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Abstract 

The combinatorics of certain osculating lattice paths is studied, and a rela- 
tionship with oscillating tableaux is obtained. More specifically, the paths 
being considered have fixed start and end points on respectively the lower 
and right boundaries of a rectangle in the square lattice, each path can take 
only unit steps rightwards or upwards, and two different paths are permitted 
to share lattice points, but not to cross or share lattice edges. Such paths 
correspond to configurations of the six-vertex model of statistical mechanics 
with appropriate boundary conditions, and they include cases which corre- 
spond to alternating sign matrices and various subclasses thereof. Referring 
to points of the rectangle through which no or two paths pass as vacancies or 
osculations respectively, the case of primary interest is tuples of paths with a 
fixed number I of vacancies and osculations. It is then shown that there exist 
natural bijections which map each such path tuple P to a pair (t, rj), where r\ 
is an oscillating tableau of length I (i.e., a sequence of l+l partitions, starting 
with the empty partition, in which the Young diagrams of successive parti- 
tions differ by a single square), and t is a certain, compatible sequence of I 
weakly increasing positive integers. Furthermore, each vacancy or osculation 
of P corresponds to a partition in r\ whose Young diagram is obtained from 
that of its predecessor by respectively the addition or deletion of a square. 
These bijections lead to enumeration formulae for osculating paths involving 
sums over oscillating tableaux. 



Keywords: osculating lattice paths, oscillating tableaux, alternating sign matrices 
2000 Mathematics Subject Classification: 05A15 



1. Introduction 



The enumeration of nonintersecting lattice paths and of semistandard Young tab- 
leaux are two basic problems in combinatorics. These problems are also closely 
related since there exist straightforward bijections between certain tableaux and 
certain tuples of nonintersecting paths. Furthermore, the problems are now well- 
understood, one reason being that a fundamental theorem, often called the Lind- 
strom-Gessel-Viennot theorem (see for example [2H Theorem 1], [23 Corollary 2] 
or [53"] Theorem 2.7.1]), enables the cardinality of a set of tuples of such noninter- 
secting paths to be expressed as the determinant of a matrix of binomial coefficients, 
thereby significantly elucidating and facilitating the enumeration. 

More specifically, the paths in this context have fixed start and end points in the 
lattice Z 2 , each path can take only unit steps rightwards or upwards, and different 
paths cannot share any lattice point. A (non-skew) semistandard Young tableaux 
(see for example [22], [51], [52] or [5U Ch. 7]) is an array of positive integers which 
increase weakly from left to right along each row and increase strictly from top to 
bottom down each column, and where the overall shape of the array corresponds to 
the Young diagram of a partition. Apart from their intrinsic combinatorial interest, 
such tableaux are important in several other areas of mathematics, including the 
representation theory of symmetric and general linear groups. Each row of a tableau 
read from right to left itself constitutes a partition, and the usual bijections between 
tableaux and nonintersecting paths (see for example [2U Sec. 6], [23 Sec. 3] or [5U 
Sec. 7.16]) essentially involve associating each row of a tableau with the path formed 
by the lower and right boundary edges of the Young diagram of that row, and 
translated to a certain position in the lattice. The condition that different paths 
cannot intersect then effectively corresponds to the condition that the entries of a 
tableau increase strictly down columns. 

It is also relevant here to consider standard Young tableaux and oscillating tableaux. 
A standard Young tableau is a semistandard Young tableau with distinct entries 
which simply comprise 1,2, ... ,n for some n, while an oscillating tableau of length I 
(see for example 0, EH E7J [58]) is a sequence of l + l partitions which starts with 
the empty partition, and in which the Young diagrams of successive partitions differ 
by a single square. It can be seen that a standard Young tableau a corresponds 
naturally to an oscillating tableau rj in which each Young diagram is obtained from 
its predecessor by the addition of a square. More precisely, if = k, then the 
Young diagram of the (/c + l)th partition of rj is obtained from that of the A;th 
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partition by the addition of a square in row i and column j. It can also be shown 
(as will be done for example in Section 18 of this paper) that a semistandard Young 
tableau r corresponds naturally to a pair (t, rj) in which t consists of the entries of r 
arranged as a weakly increasing sequence, and 77 is an oscillating tableau in which 
each Young diagram is obtained from its predecessor by the addition of a square 
(i.e., j] corresponds to a standard Young tableau). 

The primary aim of this paper is to show that these results can essentially be gener- 
alized from nonintersecting paths to osculating paths, and from pairs (t, if) in which 
the oscillating tableau r] corresponds to a standard Young tableau to more general 
pairs (t, 77) in which each Young diagram of rj can be obtained from its predecessor 
by either the addition or deletion of a square. 

More specifically, osculating paths are those in which each path can still take only 
unit steps rightwards or upwards in Z 2 , but for which two different paths are now 
permitted to share lattice points, although not to cross or share lattice edges. Such 
paths correspond to configurations of the six-vertex model of statistical mechanics 
(see for example [51 Ch. 8]). The particular case being considered in this paper is that 
in which the paths have fixed start and end points on respectively the lower and right 
boundaries of a rectangle in Z 2 . Referring to points of the rectangle through which 
no or two paths pass as vacancies or osculations respectively, the case of primary 
interest will be path tuples with a fixed number I of vacancies and osculations. It will 
then be found that there exist natural bijections which, using data associated with 
the positions of the vacancies and osculations, map any tuple P of such osculating 
paths to a pair (t, rj), referred to as a generalized oscillating tableau, in which rj is 
an oscillating tableau of length I, and t is a certain, compatible sequence of I weakly 
increasing positive integers. A feature of these bijections is that each vacancy or 
osculation of P corresponds to a partition in 77 whose Young diagram is obtained 
from that of its predecessor by respectively the addition or deletion of a square. 
If P is a tuple of nonintersecting paths, then there is such a bijection for which 
the associated generalized oscillating tableau (t, rj) corresponds to a semistandard 
Young tableau, but the overall correspondence is in fact somewhat different from the 
usual ones known between nonintersecting paths and semistandard Young tableaux. 

A summary of the bijections between tuples of osculating paths and generalized 
oscillating tableaux is given in Section 15. A particular path tuple, which is shown 
in Figure (2J is used as a running example throughout the paper, while a further ex- 
ample involving sets of path tuples is given in Section 17. An example of a tuple of 
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nonintersecting paths is given in Section 18. The bijections lead to enumeration for- 
mulae for osculating paths involving sums over oscillating tableaux, these appearing 
in Corollaries 14 and 17. 

Much of the motivation for the work reported in this paper was derived from studies 
of alternating sign matrices. An alternating sign matrix, as first defined in [38l 
[39] , is a square matrix in which each entry is 0, 1 or — 1, each row and column 
contains at least one nonzero entry, and along each row and column the nonzero 
entries alternate in sign, starting and finishing with a 1. For reviews of alternating 
sign matrices and related subjects, see for example [lOl [HI US US]- Of particular 
relevance here is that there exist straightforward bijections between alternating sign 
matrices, or certain subclasses thereof, and certain tuples of osculating paths in a 
rectangle (see for example Section 4 of this paper and references therein). Relatively 
simple enumeration formulae are known for such cases, but all currently-known 
derivations of these formulae, as given in [151 HH [351 ES, UH EH [63] , are essentially 
non-combinatorial in nature. Furthermore, it is known that the numbers of n x n 
alternating sign matrices, descending plane partitions with no part larger than n (see 
for example [JJ E31 E3, ESI EH]), and totally symmetric self-complementary plane 
partitions in a 2nx2nx2n box (see for example [21 ESI USE EH EE] ) are all equal, and 
further equalities between the cardinalities of certain subsets of these three objects 
have been conjectured or in a few cases proved, but no combinatorial proofs of these 
equalities are known. It is therefore hoped that the bijections between osculating 
paths and generalized oscillating tableaux described in this paper may eventually 
lead to an improved combinatorial understanding of some of these matters. 

Osculating paths have also appeared in a number of recent studies as a special case 
of friendly walkers (see for example [2D1 EEl Elj and references therein). However, 
all of these cases use a different external configuration from the rectangle being used 
here. In particular, the paths start and end on two parallel lines rotated by 45° with 
respect to the rows or columns of the square lattice. A general enumeration formula 
for such osculating paths has been conjectured in [9]. 

Notation. Throughout this paper, P denotes the set of positive integers, N denotes 
the set of nonnegative integers, [m,n] denotes the set {m, m + 1, . . . , n} for any 
m, n e Z, with [m,n] = for n < m, and [n] denotes the set [l,n] for any n e Z. 
For a finite set T, |T| denotes the cardinality of T. For a condition C, 5c denotes 
a function which is 1 if C is satisfied and if not, and for numbers i and j, dij 
denotes the usual Kronecker delta, 6^ = 5 i= j. For a positive odd integer n, the 
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double factorial is n\\ = n(n — 2)(n— 4) . . . 3.1, while (-1)!! is taken to be 1. 



2. Osculating Paths 

In this section, the set of tuples of osculating lattice paths in a fixed a by b rectangle, 
with the paths starting at points (specified by a subset {/3i, . . . , /3 r } of [&]) along the 
lower boundary, ending at points (specified by a subset {«i, . . . , a r } of [a]) along the 
right boundary, and taking only unit steps rightwards or upwards, will be defined 
precisely. 

For any a, b G P, the subset [a] x [b] of Z 2 will be regarded diagrammatically as a 
rectangle of lattice points with rows labeled 1 to a from top to bottom, columns 
labeled 1 to b from left to right, and being the point in row % and column j. 
The motivation for using this labeling is that it will provide consistency with the 
standard labeling of rows and columns of matrices and Young diagrams, both of 
which will later be associated with path tuples. 

The general labeling of the lattice, together with the start and end points of paths, 
is shown diagrammatically in Figure [U 
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Figure 1: Labeling of the lattice and boundary points. 



For a G [a] and (3 G [b], let H(a, b, a, (3) be the set of all paths from (a, j3) to (a, b), 
in which each step of any path is (0, 1) or (—1, 0), 



Il(a,b,a,P) := {((io,jo) = (a,P),(i 1 ,ji) 



, (llr-l,Jlr-l), {U,JLJ 



(a,b) 



{H-i,Jl- 



,U) G {(0,1), (-1,0)} for each ZG[L]} 
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where necessarily L = a — a + b— (3. It follows that \H(a,b,a, (3)\ = ^ 



a — a + b— (3^ 



a — a 



For a, a' G [a] and (3,(3' G [6], with a < a' and /3 < (3', paths P G II(a, b, a, (3) and 
P' G Il(a, 6, a', /?') are said to be osculating if they do not cross or share lattice edges, 
but possibly share lattice points. More precisely, this means that if P t = P(, = (i,j) 
for some / and /' (which implies that / = a — i+j — (3, I' = a — i + j — /3'), then 
P M = P m = PU = + (if I' ± 0) and P{, +1 = + (if 

I' 7^ a— a'+b— (3'). Any such common point will be referred to as an osculation 
of P. 

For r G [0, min(a, b)], a = {a±, . . . , a r } C [a] and (3 = {(3±, . . . , (3 r } C [b], with 
cei < . . . < a r and (3\< . . . < (3 r , let OP(a, 6, a, /3) be the set of r-tuples of pairwise 
osculating paths in which the fc-th path is in U(a, b, atk, (3k), 

OP(a,b,a,(3) := |P = (P u . . . , P r ) G U(a, b, a u A) x . . . x U(a, b, a r , (3 r ) 

, (2) 

Pk and Pfc + i are osculating for each k G [r — 1] | . 

Also, for any a, b G P, let BP(a, 6) be the set of all pairs (a, (3) of boundary points, 
BP(a,fo) := {(a,/5) | a C [a], (3 C [6], |a| = \(3\) . (3) 

min(a,6) . . . , . . , , 

/ a+o 



It follows that |BP(a,6)| = £ Q Q = 



Throughout the remainder of this paper, a and b will be used to denote positive 
integers, corresponding to the dimensions of a rectangle of lattice points, and (a, (3) 
will denote an element of BP (a, b). 

Now let OP (a, b) be the set of all tuples of osculating paths in [a] x [b] with any 
boundary points, 

OP(a,6):= |J OP(a,b,a,(3). (4) 

(a,/3)GBP(a,fc) 

For P G OP (a, b), any point G [a] x [b] through which no path of P passes will 
be referred to as a vacancy of P. Define N(P) to be the set of all vacancies of P, 
X(P) to be the set of all osculations of P and x(P) to be the number of osculations 
of P, 

X (P):=\X(P)\. (5) 
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A tuple of noninter seating paths is any P for which X(P) = 0. Nonintersecting 
paths will be considered in more detail in Section 18. 

Define also a vacancy-osculation of P G OP(a, b) as either a vacancy or osculation 
of P, and the vacancy-osculation set Z(P) as the set of all vacancy-osculations of P, 



Z(P) ■= JV(P)UI(P) 



(6) 



In other words, Z(P) is the set of points of [a] x [b] through which either zero or 
two paths of P pass. It will be of particular interest to consider sets of path tuples 
with I vacancy-osculations, for fixed I G N, 



OP(a,&,Z) := {P G OP(o,6) | \Z(P) \ = /} 
OP(a,b,a,P,l) := {P G OP (a, 6, a, (3) | \Z(P)\ = /} 



(7) 



a primary aim of this paper being to study the properties and cardinality of OP (a, b, 

a,(3,l). 

Finally, note that there are trivial bijections, involving reflection or translation, 
between certain sets of path tuples. More precisely, using « to denote the existence 
of a bijection between sets, 

OP {a, b, a, (3,1) w OP(6, a, (3, a, I) 

- (8) 

~ OP(a+a, b+b, {a+cti, . . . , a + a r }, {b+fii, . . . , b+j3 r }, l + ab+ab+ab) , 

for any a, b G P, a, 6 G N and (a, /3) = ({«i, • • . , a r }, {/3i, . . . , /3 r }) G BP(a, 6). For 
the first bijection of (jHJ) each path is reflected in the main diagonal of the lattice, 
while for the second bijection of ([SJ) each path is translated by (d,b). 

An example of an element of OP(4, 6, {1, 2, 3}, {1, 4, 5}, 11) is P = (((4, 1), (3, 1), 

1 2 3 4 5 6 



I 



4, * 



Figure 2: Example of a tuple of osculating paths. 
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(3,2), (3,3), (3,4), (2,4), (2,5), (1,5), (1,6)), ((4,4), (3,4), (3,5), (2,5), (2,6) 
4,5), (4,6), (3,6))), which is shown diagrammatically in Figure [2J 



For this case, N(P) = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 1), (2, 2), (2, 3), (4, 2), (4, 3)}, 

X(P) = {(2, 5), (3, 4)} and x(-P) = 2. This will serve as a running example through- 
out this paper. 

3. Edge Matrices 



In this section, it will be seen that each tuple of osculating paths corresponds natu- 
rally to a pair (H, V) of {0, 1} matrices, which will be referred to as horizontal and 
vertical edge matrices. 

For P G OP (a, b, a, (3), the correspondence is given simply by the rule that 
is or 1 according to whether or not P contains a path which passes from (i, j) 
to + and that Vij is to 1 according to whether or not P contains a path 
which passes from + to Thus is associated with the horizontal 

lattice edge between and and Vy is associated with the vertical lattice 

edge between (i,j) and + It is also convenient to consider boundary edges 
horizontally between (i, 0) and (i, 1), and between (i, b) and (i, 6+1), for each i G [a], 
and vertically between (0, j) and (1, j), and between (a, j) and (a + 1, j), for each 
j G [b], and to include in each path P k the additional points (a + 1, (3k) at the start 
and (ctk,b + l) at the end. Each point (i,j) G [a] x [b] can then be associated with 
a vertex configuration which involves the four values Vij, Hij and V^ij, this 

being depicted diagrammatically as 



v i-l,j 



(9) 



It can be seen that for any tuple of osculating paths there are only six possible path 
configurations surrounding any lattice point, given diagrammatically as: 



(10) 



Correspondingly, there are six possible vertex configurations: 



The numbers below each vertex configuration will be used to label the six possible 
types. Thus, type 1 corresponds to an osculation, and type 2 to a vacancy. 

It can also be seen that the six cases of (ITTj) correspond exactly to the simple but 
important condition 



Hi.n-i + Vij — Vi-\ j + H., 



y ■ 



'121 



for each G [a] x [b\. 



Accordingly, taking into account all of the previous considerations, sets of edge 
matrix pairs for a, b G P and (a, (3) G BP(a, b) are defined as 

EM(a, b) := {(H,V)\ • H and V are matrices with all entries in {0, 1} 

• H has rows labeled by [a], columns labeled by [0, b] 

• V has rows labeled by [0, a], columns labeled by [b] (13) 

• H i0 = for all % G [a] , V 0j = for all j G [b] 

• Hij-i + = VUj + Hij for all G [a] x [6] } 



and 

EM{a,b,a,P) := 
{(H,V) G EM(a,6) 



;6 



s 



for all i6 [a], V a j = for all j G [6]} . 



(14) 



It can be seen that the 'boundary conditions' on (H,V) G EM (a, b, a, 0) are that 
the first column of H and first row of V are zero, and that the last column of H 
and last row of V are specified by a and f3 respectively. 



As already indicated, for any P = (Pi, . . . , 
(H, V) G EM(o, 6, a, (3) is given by 



P r ) G OP(a, 6, a, /?), a corresponding 



y 



1, (P fc )i = (z, j) and (P fc )/+i = (i, J + l) for some k and /, 
or i G a and j = b 

0, otherwise 



(15a) 
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{1, (P k )i = and (P k )m = for some k and I, 

or i = a and j G (3 (15b) 
0, otherwise , 

and it can be seen straightforwardly that this mapping is a bijection between OP(a, b, 
a, (3) and EM(a, 6, a, 

It can also be seen that the number of osculations of P can be expressed in terms 
of the corresponding (H, V) as 

x(p) = £ //,. ; ,r„ = £ i; :., //„ . (i6) 

(i,i)e[a]x[6] (i,j)e[a]x[b] 

since (z, j) is an osculation if and only if H it ^i = Vij = 1, or equivalently Vi-\j = 
Hij = 1. 



Returning to the running example, the edge matrices are 

/Voi . . . V e\ 



(H 10 . 


■H 16 \ 






■ #46/ 





1 1 1 
11110 1 



\v 41 ...v m ) 



/o o\ 

1 

1 1 

10 10 1 

\1 1 1 0J 



■ (17) 



The edge matrix representation of osculating paths corresponds to the standard rep- 
resentation of configurations of the six-vertex or square ice lattice model in statistical 
mechanics (see for example Ch. 8] and references therein). In this model = 
is represented by a leftward arrow on the corresponding lattice edge, = 1 by a 
rightward arrow, = by a downward arrow, and = 1 by an upward arrow. The 
osculating paths then follow the rightward and upward arrows, and condition ([12]) 
corresponds to arrow conservation at each lattice point (i.e., the numbers of arrows 
into and out of each point are equal). One of the main quantities of interest for such 
statistical mechanical models is the partition function, which is a certain weighted 
sum over the configurations of the model. The particular case being considered here 
is that of configurations of the six-vertex model on an a by b rectangle with fixed 
boundary conditions in which on the upper boundary all arrows point down, on the 
left boundary all arrows point left, on the lower boundary arrows point up or down 
according to whether or not their position is in (3, and on the right boundary arrows 
point right or left according to whether or not their position is in a. Note that the 
six- vertex model has been extensively studied with a variety of boundary conditions. 
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See for example [3 Ch. 8 &; 9] for the details of studies with periodic (i.e., toroidal) 
boundary conditions, and [U [311 H21 EH EI] for some studies with other boundary 
conditions. 



4. Alternating Sign Matrices 

In this section, it will be seen that each tuple of osculating paths also corresponds 
naturally toa{ — 1,0,1} matrix, which will be referred to as an alternating sign ma- 
trix. Although this representation of osculating paths will not be used in obtaining 
any of the results of this paper, it is introduced here in order to present some known 
formulae for the cardinality of special cases of OP (a, b, a, 0) which are usually given 
in the context of such a representation. Also, as indicated in Section 1, much of the 
motivation for the work reported in this paper was derived from the studies of alter- 
nating sign matrices in which these formulae were obtained. Note, however, that the 
enumeration results obtained later in this paper apply to cases of OP(a, b, a, j3, 1), 
i.e., for which the path tuples all have / vacancy-osculations, whereas the enumer- 
ation formulae listed in this section apply to certain cases of OP(a, b, a, /3), i.e., for 
which there is no restriction on the number of vacancy-osculations. Note also that 
the alternating sign matrices defined in other papers comprise a special case of the 
alternating sign matrices defined here, and so will be referred to here as 'standard 
alternating sign matrices'. 

For any a, b G P and (a, (3) G BP(a, b) define the associated set of alternating sign 
matrices as 



For any (if, V) G EM(a, b, a, (3), a corresponding A G ASM(a, b, a, (3) is given simply 




(18) 



• E?=i^i = <W for all j G lb}}. 



by 



-An 




(19) 
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or equivalently, due to (Jl2"]l . 



(20) 



for each (i, j) G [a]x[6] (i.e., A is the column difference matrix of H and row difference 
matrix of V). Note that under this mapping vertex configurations 1-4 of (TTTj) at 
give Aij = 0, configuration 5 gives Aij = —1 and configuration 6 gives A^ = 1. 

It can be checked that the mapping (I19II20I) is a bijection between EM(a, 6, a, /3) and 
ASM(a, 6, a, and that the inverse mapping is 



Hij = E A^' , for each G [a] x [0, b] 
j'=i 

i 

Vij = A'j , for each G [0, a] x [b] 
i'=l 



(21) 



(i.e., H and are respectively the partial column and row sum matrices of A). 

It follows, using ffTBT) and (T2"TT) . that the number of osculations of any P G OP (a, b) 
can be expressed in terms of the corresponding A G ASM(a, b) as 



x(p) = E 

{(M'jj')eH 2 x[t>] 2 

*'>i,i'<i} 



A^ Ay ji 



E 

{(i,i'jj')6[a]2 x [6] 2 | 

i'>i,j'<j} 



(22) 



The alternating sign matrix for the running example is 



.4 



/0 1 \ 

1 

10 0-1 1 

\fl 1 -1/ 



(23) 



Five previously-studied cases of alternating sign matrices will now be considered, 
for any ?i6P: 

• ASM(n,n, [n], [n]) 

• ASM(n,n+l, [n], [n+l]\{n+l— m}), for m G [0, n] 

• ASM(n, n+m, [n], [n— 1] U {n+w}), for m G N 

• ASM(n, 2n-l, [n], {1, 3, 5, ... , 2n-l}) 

• ASM(n, n, {1, 3, 5, . . . , 2 [f ] - 1}, {1, 3, 5, . . . , 2 [f ] - 1}) 
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It can be seen that the third case reduces to the first case for m — 0. 

The elements of ASM(n, n, [n], [n]) will be referred to here as standard alternating 
sign matrices. They are simply nxn { — 1,0,1} matrices in which along each row 
and column the sum of entries is 1, and the nonzero entries alternate in sign. They 
were introduced in [38| 139], in which an enumeration formula was conjectured which 
gives 

|OP(n,n,[n],[n])| = Hf^- (24) 

This was eventually proved in [62] and, using a different method, [35]. The corre- 
spondence between standard alternating sign matrices and edge matrices was first 
identified in [17], and is also discussed, at least in the statistical mechanical model 
version, in [8], [19l S3] • The correspondence between standard alternating sign ma- 
trices and osculating paths is also considered in Sec. 5], 0, Sec. 2], [T5J Sec. 9] 
and [591 Sec. IV]. 

The six-vertex model boundary conditions for this case, in which all arrows on the 
upper and lower boundaries point into the square, and all arrows on the left and right 
boundaries point out of the square, are known as domain wall boundary conditions 
(see for example [31]). 

It can be seen that the nxn permutation matrices are included in ASM(n, n, [n], [n]), 
and, from (|2"2"|) . that if A is a permutation matrix which corresponds to P G 
OP(n, n, [n], [n]), then the number of osculations x(P) ls simply the inversion num- 
ber of A. The inversion number of any A G ASM(a, b) could be defined as 1(A) = 
^{(Myj')e[o] 2 x[6] 2 \%>>i, j'<j} A-ij Aiiji, this being consistent with a definition of inver- 
sion number for standard alternating sign matrices given in [5H]. It then follows 
(see for example [17J Theorem 2c]) that 1(A) = x(P) + where P G OP(a, b) 

corresponds to A, and n(A) is the number of —1 entries in A. 

It can be seen that in any standard alternating sign matrix, there is a single 1 in 
each of the first and last row and column. Furthermore, using elementary symmetry 
considerations, it follows that the number of (n+1) x (n+1) standard alternating 
sign matrices A with Aij = 1 is the same for (i, j) taken to be any of the eight cases 
(l,m+l), (1,71+1— m), (m + 1,1), (n+1— m, 1), (m+l,n+l), (n+1 — m, n+1), 
(n + l,m+l) or (n+1, n + 1 —m), for fixed m G [0, n]. Focussing on the case 
(n + 1, n + 1— m), i.e., matrices with the 1 of their last row in column n + 1 — m, so 
that An+ij = 5j /n+ i_, m , it follows from (I2T1) that H^ij = f^^i-m, and then from 
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= 1 and ffl~2]) that V n j = 5j^. n ^i- m , which implies that the submatrices formed 
by the first n rows of such A comprise ASM(n, n+1, [n], [n+l]\{n+l— m}). Obtaining 
the cardinality of this and the seven related sets is known as refined alternating sign 
matrix enumeration, and, following conjectures in [55], was achieved in [63J and 
more recently in [151 EIJ giving 

i™/ irir m r 1 m (2n-m)! (n+m)! A (3i-2)! , . 

OP n,n+l, n n+l n+l-m = v IlVx^u ' 25 

n! m! (n— m)! (n+z)! 



The related case of ASM(n, n + m, [n], [n— 1] U {n + m}) was considered in [21], in 
which an enumeration formula gives 



\OP(n,n+m, [n], [n — l]U{n+m})\ 



n—1 , , ■ 

£|OP(n-l,n, [ n _l],[ n ]\ {n _i})|^"- r/ 



i=0 



(26) 



1 !j__ 2 (3i+l)! ^ (2n-2-i)! (n-l+z)! (m+i)! 



(n— l)!m! (n+z)! ~^ i\ 2 (n — l—i)\ 



A matrix is horizontally- symmetric if it is invariant under reflection about the hor- 
izontal line through its center, and horizontally- and-vertically symmetric if it is 
invariant under reflection about both the horizontal and vertical lines through its 
center. It can be seen that for standard alternating sign matrices, such symmetries 
can only occur for odd-sized matrices. 

Considering horizontally-symmetric (2n+l) x (2n+l) standard alternating sign ma- 
trices A, i.e., those for which Aij = ^rt+a-y for each (i,j) G [2n+l]x [2n + l], 
it can be seen that the first and last columns always have their l's in the mid- 
dle row, An = A it 2n\-i = 5i,r»fij and that the middle row always consists entirely 
of alternating l's and —l's, Ai+ij = ( — l)^*" 1 - Proceeding to the corresponding 
edge matrices, it is then found that along the first column of vertices H iQ = 0, 
Vn = <5j>ri+-i and Hn = di^i, along the last column of vertices H^n = <^n+i, 
Vi,2nH = 5i>nu and Hi,2n\\ = 1, and along the middle row of vertices V n j = 6 j even , 
H^i j = Sj dd and Ki+ij = 8j odd- It follows that each such A is uniquely deter- 
mined by the submatrix formed by the first n rows of A with their first and last 
columns deleted (i.e., A^ with (i,j) G [n] x [2, 2n]), and that these submatrices 
comprise ASM(n, 2n — 1, [n], {1,3,..., 2n— 1}). Following conjectures in [I5J H6], an 
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enumeration formula was proved in (36J giving 

|OP(n,2n-l,[n],{l,3,...,2n-l})| = f[ . (27) 

Considering horizontally- and- vertically-symmetric (2n + 3) x (2n + 3) standard al- 
ternating sign matrices A, i.e., those for which Aij = Ain\4-4,j = Ai t2n +4^j for 
each G [2n + 3]x [2n + 3], it follows, since both A and A 1 are horizontally 

symmetric, that each such A is uniquely determined by the submatrix formed 
by with G [2,72+1] x [2, n + 1], and that these submatrices comprise 

ASM(n, n, {1, 3, . . . , 2[f ]-l}, {1, 3, . . . , 2 [§ ]-l}). Following conjectures in 051 E^, 
an enumeration formula was proved in [H] giving 

|OP(n,n,{l,3,...,2[f]-l},{l,3,...,2[fl-l})| = 

(LjJ + 1)! A (3Q ! (28) 
3LfJ (2n+l)! LfJ! M (n+i)! ' 

5. Vacancy-Osculation Sets and Matrices 

In this section, the vacancy-osculation set Z(P) associated with each path tuple 
P G OP (a, b) will be studied further, and it will be shown that, for fixed a and b, P 
is uniquely determined by Z(P). 

For any a, b G P, (a, (3) G BP(a, b) and I G N, define sets of vacancy-osculation sets 
as 

VOS(a, b) := Z(OP(a, 6)), VOS(a, 6, a, /3) := Z(OP(a, 6, a, /?)), 

(29) 

VOS(a,6,Z):=Z(OP(a,6,Z)), VOS(a, b, a, (3,1) := Z (OP (a, b, a, (3,1)) , 
where for each subset Q of OP(a, 6), Z(Q) := {Z(P) \P eQ}. 

It is sometimes convenient to represent or visualize a vacancy-osculation set S G 
VOS(a,6) as a vacancy-osculation matrix M(S). This is an a x 6 {0,1} matrix 
defined simply by 

M{S)ij : = 5 {itj)eS , for each G [a] x [b] . (30) 

Note that in a vacancy-osculation matrix, the positions of the nonzero entries cor- 
respond to the positions of vertex configurations of types 1 and 2 in (jlip . whereas 



15 



in an alternating sign matrix they correspond to the positions of configurations of 
types 5 and 6. 

By examining the six possibilities (TlTT) for each vertex configuration ([9]), it can be 
seen that if S = Z(P) is the vacancy-osculation set of P G OP(a, b), and (H, V) G 
EM(a, h) is the edge matrix pair which corresponds to P, then 

M(S) tJ = 5 H ,^ uVij = 5 v ^ hHi , . (31) 



The vacancy-osculation set and matrix for the running example are 

S = {(1, 1), (1, 2), (1, 3), (1, 4), (2, 1), (2, 2), (2, 3), (2, 5), (3, 4), (4, 2), (4, 3)} (32) 
and 



M{S) 



/l 1 1 1 0^ 

1110 10 

1 

Vo i i o o oy 



(33) 



Lemma 1. Each tuple of osculating paths in OP (a, h) is uniquely determined by its 
vacancy-osculation set. 



In other words, ([6]) gives an injective, and hence by (T291 bijective, mapping from 
OP (a, b) to VOS(a,5). It immediately follows that OP(a, b, a, (3), OP(a,6, 1) and 
OP(a, b, a, (3, 1) are bijectively related to VOS(a, b, a, (3), VOS(a, b, I) and VOS(a,6, 
a, (3,1) respectively, for any (a, j3) G BP(a, b) and / G N. 

Proof. The relations of fl3TT) can also be written 

Vij = Sh^^m^h and Hij = 5v^,M{s) tj i (34) 

which can be regarded as recursion relations for (H,V). Therefore, if S is the 
vacancy-osculation set of P G OP(a, b), and (H,V) G EM(a, b) is the edge matrix 
pair which corresponds to P, then (H, V), and hence P, can be uniquely recovered 
from S using (|34"|) together with the initial conditions H i0 = Vqj = for all i G [a], 
J e [6]. □ 

In fact (13TI) or (jM|) can also be written 

+ = V itH . + Hij = 5 (lMS = M(S) tJ + 1 (mod 2) , (35) 
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so that, using this with the initial conditions, (H, V) can be expressed explicitly in 
terms of S as 



Hi A = 



{(i-k,j-k) | ke[0,imn(i,j)-l]} U 

{(i-k-l,j-k) | fce[0,min(i-U)-l]})\ S 



min(ij') — 1 min(i— — 1 

k=0 k=0 

for each G [a] x [0, b] 



{(i-k,j-k) | fce[0,min(i, j)-l]} U 

{(i-k,j-k-l) | A;G[0,min(z,i-l)-l]})\ S 



(mod 2) 



(mod 2) 



(36a) 



(mod 2) 



(mod 2) 



(36b) 



min(ij') — 1 min(i ,j— 1) — 1 

= 5i>j + M{S)^ k + J2 Mffli-kj-M 

k=0 k=0 

for each G [0, a] x [6] , 
where each and V^j is taken to be or 1. 

It can be seen that for any a, b G P, and [a] x [6] are both in VOS(a, b), cor- 
responding to the path tuple P G OP(a, 6, [min(a, &)], [min(a, &)]) in which path p 
passes vertically from (a,i) to and then horizontally from to (z,fo), for 

each i G [min(a, &)], e.g., 



and [a] x [6] corresponding to the single empty path tuple in OP(a, b, 0, 0), e.g., 



Finally, for any S G VOS(a,6) corresponding to P G OP(a, b), a vacancy or oscu- 
lation of P will also be referred to as a vacancy or osculation of S, and the same 
notation will be used for the sets of vacancies and osculations, X(S) := X(P) and 
N(S) := N(P), and for the number of osculations, x(S) '■= x{P)- 
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6. Partitions 



In the previous sections, four bijectively-related sets, OP (a, 6), EM(a, b), ASM(a, b) 
and VOS(a, b), have been described, and the aim of forthcoming sections will be to 
obtain a further, intrinsic characterization of VOS(a, b) which facilitates the enu- 
meration of these sets. The first step in this process will involve a transformation 
between a boundary point pair (a,/3) G BP(a, b) and a partition, so in this section 
the relevant notation for partitions is outlined. 

A partition A = (Ai, A2, . . .) is an infinite sequence of nonnegative integers in weakly 
decreasing order, Ai > A2 > . . ., which has only finitely-many nonzero terms. The 
nonzero terms are the parts of A, and the number of parts is the length of A, de- 
noted ^(A). If the sum of parts is n, then A is said to be a partition of n, denoted 
|A| — n. The set of all partitions will be denoted as Par. When writing a partition 
as an explicit sequence of terms, some or all of the zero terms will be omitted. The 
unique partition of zero will be denoted by and called the empty partition. 

For any partition A define 

Y(X):={(t,j)e¥ 2 \t<£(X),j<X l }. (37) 

The Young diagram of A is then a depiction of A in which a unit square is centered at 
each G Y(X), using matrix-type labeling of the rows and columns of the lattice. 
The conjugate of A, denoted A*, is the partition whose Young diagram is related to 
that of A by reflection in the main diagonal, Y(X t ) = \ G Y(X)}. It 

follows immediately that |A| = | A* j = |Y(A)|, £(A*) = A x and A" = A. 

A running example will be the partition A = (3,2,2), for which |A| = 7, i(X) = 3, 
A* = (3, 3, 1), Y(X) = {(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (3, 1), (3, 2)}, and the Young 
diagram is 



The rank of a partition A is defined as 

p(A):=|{ie[^(A)]|A i >i}|, (38) 
and is thus the number of squares on the main diagonal of the Young diagram of A, 

p(\) = \{(i,j)eY(\)\i = j}\. 
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The Frobenius notation for a partition A is 

A = (71, . . . ,7 r I 81, . . . ,S r ), where r = p(A), 7* = \-i and ^ 

5i = A' — « for each i G [r] . 

Thus, 7i is the number of squares in row % of the Young diagram to the right of the 
main diagonal, and 8i is the number of squares in column % of the Young diagram 
below the main diagonal. It can be seen that each pair of tuples of nonnegative inte- 
gers (71, ... , 7 r ) and (81, . . . ,8 r ), with 71 > . . . > 7 r and 81 > . . . > 5 r , corresponds 
to a unique partition (71, . . . , j r \ Si, . . . , 8 r ) with rank r. For the example (3, 2, 2), 
the rank is 2, and the Frobenius notation is (2, | 2, 1). 

For any G Z 2 , let the content of or of a unit square centered at be 
j — i. Then, for any d G Z and any subset T of Z 2 , let the d-diagonal of T be the 
set of points of T with content d, 

D d (T):={(i,j)eT\j-i = d}, (40) 

and let the d-rank of T be the number of points in the ci-diagonal of T, 

R d (T) := \D d (T)\. (41) 

For a partition A, define 

p d (X) := R d (Y(X)). (42) 
It can be seen that po(A) = p(A) and 

. . (\{i£[pW]\Si>-d}\, d<0 

PdM = < 43 
ll{*e[p(A)]|7i>4l, ^>0, 

where (71, ... , r ) p {\) \8\, . . . , 5 p (a)) is the Frobenius notation for A. For the running 
example, 

1, {-2,1,2} 
p,(3,2,2)= 2, de{-l,0} (44) 
0, otherwise . 

Partitions A and jj, will be said to differ by a square, denoted A ~ /i, if and only if 
there exists % G P such that | — A*fc I — $ki f° r each k G P. Partitions A and /j, thus 
differ by a square if and only if there exists G P 2 such that Y(X) is the disjoint 
union of Y(p) and {(i, j)}, or Y(/i) is the disjoint union of Y(X) and {(i, j)}, and 
in such a case the diagonal difference between A and \x is defined as 

A(\,n):=j-i. (45) 
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In other words, for A~/i, A(A, p) is the content of the square by which the Young 
diagrams of A and p differ. It can be seen that given a partition A and a positive 
integer i, there exists a partition p with A~/i and Y(X) = Y(p)U{(i, Aj)} if and only 
if Aj > A»fi, and there exists a partition p with A~/i and Y(/i) = F(A)U{(i, Aj + 1)} 
if and only if % = 1 or Aj_i > Aj. It follows that for any partition A, the number of 
partitions p with A~/i is 2£(A) + 1, where £(X) is the number of distinct parts of A. 
It can also be seen that for fixed A, each p with A~/i is uniquely determined by the 
diagonal difference A(A,/i). 

Define a change diagonal of a partition A to be any integer d for which there exists 
a (necessarily unique) partition p with A ~ p and A(A,/i) = d It can be checked 
straightforwardly that 

d is a change diagonal of A if and only if 

f (1,0) or (0,1), d<0 
(p d (X)-p^(X),p m (X)-p d (X)) = (1,-1) or (0,0), d = 

[(0,-1) or (-1,0), d > 0. 

Within each of the three cases of d in (T4"6]) . the first and second alternatives cor- 
respond to the existence of p ~ A with respectively |/i| = |A| — 1 and \p\ = |A| + 1. 
For the example A = (3, 2, 2), the change diagonals are —3, —1, 1, 2 and 3, and the 
corresponding partitions /i~A are (3,2,2,1), (3,2,1), (3,3,2), (2,2,2) and (4,2,2) 
respectively 

Finally, for any a, b G P, define Par(a, b) to be the set of all partitions whose Young 
diagram fits into an a by b rectangle, 

Par(a, b) := {A G Par | £(X) < a and A x < b} . (47) 

By associating a partition in Par(a, b) with the path formed by the lower and right 

boundary edges of its Young diagram, it can be seen that |Par(o,6)| = ( a ^)- For 

any A G Par(a,6), the (a, b)- complement of A, denoted [a] x [6]\A, is defined to be 
the partition 

[a]x[6]\A := (6-A a ,6-A _ 1 ,...,6-A 1 ), (48) 
where, as always, Aj is for i > £(X). It can be seen that 

Y([a]x[b]\X) = {( a +l-t,b+l-j)\(t,j)e[a]x[b]\Y(X)} (49) 

and 

p d ([a]x[6]\A) = R d ([a]x[b\) - p M (X) . (50) 
An example of a complement of A = (3, 2, 2) is [4] x [6]\A = (6, 4, 4, 3). 
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7. Correspondence Between Boundary Point Pairs and Partitions 

Returning to the case of osculating paths OP(a, b), it will be shown in this section 
that each boundary point pair in BP(a, b) corresponds naturally to a partition in 
Par (a, b), and that the correspondence leads to an important property for osculating 
paths. 

For any (a, (3) = ({cm, . . . , a r }, {Pi, . . . , /3 r }) G BP(a, b), the corresponding partition 
is defined to be 

K,b,a,p ■= [a] x [b]\(b-Pi, b-p r \a-ai,..., a-a r ) , (51) 

where the definitions fl39|) of Frobenius notation and (T4*8l) of the (a, 6)-complement 
for partitions are being used, and ai < . . . < a r , (3i < . . . < (3 r . 

It can be checked straightforwardly that for fixed a and b this mapping is a bijection 
between BP (a, b) and Par(a,6), and that for A G Par (a, b) the inverse mapping is 
given by 

(a a ,6,A, /3 a ,b,x) = ({a-Si, . . . ,a-8 r }, {6-71, • • • ,b-j r }) , 

(52) 

where [o]x[6]\A = (71, . . . , j r \ 6 1} . . . , S r ) . 
It can also be seen that 

r 

h,a,f},a = {K,b, a ,py , \K,b,a,f}\ = ab ~ (a+b+l)r + ^(«i+ /3j) , (53) 

1=1 

and that some special cases of f)5ip are 

A a ,6,[min(a,6)],[mm(a,fe)] = 0, ^a,fe,0,0 = {b, ■ ■ ■ ,b) 

a 

K,b,[a],P = (Pa — a, ■ ■ ■ ifa — 2, — 1) , ^o,6,a,[6] = («6 _ b, . . ., a 2 — 2, «i — 1) 

A a ,a,a,/3 = (a— «i, . . . , a— a? \ a— (3i, . . . , a— /3 f ) , 

where {ai, . . . , a f } = [a] \a , {^1, . . . ,^ P } = [a]\/3 . 

For a = 4, 6 = 6 and (a, /9) = ({1, 2, 3}, {1, 4, 5}), as used for the running example 
of a tuple of osculating paths, the corresponding partition is A4 j 6,{i,2,3},{i,4,5} = 
[4] x [6] \ (5, 2, 1 1 3, 2, 1) = [4] x [6] \ (6, 4, 4, 3) = (3, 2, 2), which was used as the 
running example of a partition. 

For the five special cases of a, b, a and /3 used in Section 4 to give previously-studied 
cases of alternating sign matrices, the corresponding partitions are given in Table [TJ 
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a, b, a, (3 




,1) 
,1) 



n, n, {1,3, . . . ,2ffl -1}, {1,3, ... ,2^1-1} (n-l,n-2, 



Table 1: Partitions for alternating sign matrix cases. 



Lemma 2. For any tuple of osculating paths P G OP (a, b, a, 0) and integer d, 



Here, N(P) and X(P) are the sets of vacancies and osculations for P as defined in 
Section 2, and Rj and pd are d-ranks as defined in (HT1) and (jl2l . In other words, 
the difference between the numbers of vacancies and osculations on any diagonal of 
the lattice is independent of the path tuple P, and equal to the number of squares 
on the corresponding diagonal of the Young diagram of \ a ,b,a,p- 

Lemma 2 implies that for a path tuple P G OP(a, b, a, 0), the partition \ a ,b,a,p 
can be obtained geometrically by first marking on the lattice the vacancies N(P), 
and marking differently the osculations X(P), then along each diagonal -Dd([a] x [b]) 
removing all of the osculation marks together with an equal number of vacancy 
marks, and finally along each diagonal moving, if necessary, the remaining vacancy 
marks diagonally upwards and leftwards so that they occupy adjacent positions 
starting from the upper or left boundary of the lattice. The occupied points then 
correspond to Y (X a ,b,a,p) ■ 

Proof. As usual, let (a, 0) = ({ai, . . . , a r }, {Pi, . . . , (3 r }) with a± < . . . < a r and 
/?!<...< p r . Now let Jd(a, b, a, (3) be the number of paths of P which pass 
through the (^-diagonal x [b]). Since path Pi passes continuously by upward 

or rightward steps from (a, Pi) to it can be seen by placing a line through 

the points of D d ([a] x [&]) that J<f(a, b, a, 0) is the number of start points {a, Pi) on 
or to the left of (a,a + d) for d < b — a, and the number of end points (aj, b) on or 
above (b — d,b) for d > b — a. Therefore, as suggested by the notation, Jd(a,b,a, 0) 



R d (N(P)) - R d (X(P)) = Pd (\ a , b , a ,p) . 



(55) 
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is independent of P G OP(a, b, a, j3), and given explicitly as 

(\{i G [r] | ^ < a+d}\, d<b-a 
J d (a,b,a,P) = I (56) 
[|{z G [r] | Oi < b-d}\, d>b-a. 

Denning I(P) to be the number of points of [o]x[6] through which a single path of P 
passes (i.e., points with vertex configurations of types 3-6 in (fTTI) ). so that [a]x[6] is 
the disjoint union of PyP), X(P) and N(P), and using the fact that an osculation 
is a point through which two paths pass, it follows that 

R d ([a] x [6]) = Rd(I(P)) + R d (X(P)) + R d (N(P)) 

57 

J d (a, b, a, (3) = R d (I(P)) + 2R d (X(P)) . 

Eliminating J d (a,b,a, 0) and R d (I(P)) from (1561) and fl571) now gives 

e [r] I A < a+d}|, d < 6-a 
RiWPy-RiiXiP)) = R d ([a]x[b]) - { (58) 

[|{« G [r] | otj < d > b-a. 

Finally, using (1501) with A = (6— Pi, . . . , 6— /3 r | a— act, . . . , a—a r ), and then using (113"]) 
on the second term from the RHS of (1501) . it is found that the RHS of (j58p is 
Pd([o] x [b]\(b—j3i, . . . , b— j3 r | a— a%, . . . , a — a r )) as required. □ 

It follows from Lemma 2, by summing (|55l) over all ci G [1 — a, b — 1], that 

|iV(P)| = |A a , w |+x(P), for any Pe OP (a, b, a, /3) , (59) 

and therefore that 

|S| = |A„ )W | + 2x(-S), for any 5g VOS(a, 6, a, /3) . (60) 

Note that in (|59|) or (|60|) . |A 0) 6 iQi)i g| can be obtained directly from a, b, a and (3 
using (155|) . 



Lemma 2 can be verified for the running example by observing that 

R d (N(P)) = < 
and that Pd(A 4 ,6,{i,2,3},{i,4,5}) is given by 



fl, rfG{-2,2,3} 
2, de {-1,0,1} and R d (X(P)) 
0, otherwise 



1, de{l,3} 

0, otherwise , 
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8. Vacancy-Osculation Sets which Differ by a Vacancy-Osculation 

In this section, the relationship between two vacancy-osculation sets which differ 
by a single element will be examined, and it will be seen that the corresponding 
partitions then differ by a square. 

Vacancy-osculation sets S,S' G VOS(a,6) will be said to differ by the vacancy- 
osculation if and only if the sets differ by the single element (i, j), i.e., 5" is 
the disjoint union of S' and or S' is the disjoint union of S and 

Furthermore, in such a case the diagonal difference between S and S' will be defined 

as 

A(S,S') :=j-i. (61) 

The actual position of (i,j) will sometimes be unimportant, so S,S' G VOS(a, b) 
will be said simply to differ by a vacancy-osculation, denoted S ~ S', if there exists 
some G [a]x[&] such that S and S' differ by It can be seen that ~ and A 

for vacancy-osculation sets have been defined analogously to ~ and A for partitions, 
and the reason for this will become apparent in Lemma 6. 

In terms of the associated vacancy-osculation matrices, S,S' G VOS(a, b) differ by 
a vacancy-osculation if and only if the entries of the matrices match at all positions 
except a single G [a)x[b], \M(S) kl -M(S') k i\ = %j),(fc,j) for each (k,l) G [a]x[b]. 

Note that if vacancy-osculation sets S and 5" differ by a vacancy-osculation, it 
does not necessarily follow that N(S) and N(S'), or X(S) and X(S'), individually 
differ by a single element, it being possible that N(S) fl X(S') or X(S) fl N(S') are 
nonempty. 

Define a deletion point of S G VOS(a,6) to be any G S for which there exists 
S' G VOS(a, b) with S ~ 5" and S' = S and define an addition point of 

S G VOS(a,6) to be any G [a] x[b]\S for which there exists S' G VOS(a,6) 
with S ~ S' and S' = S U {(i, j)}. Define a change point of S to be any deletion 
point or addition point of S. 

For the running example, with S given by fl32|) . it can be seen in Figure [3] and 
Table[2]that S has deletion points (4,3), (2,3), (3,4), (1,4) and (2,5), and addition 
points (4,5), (3,5), (4,6) and (3,6). In Figure the vacancy-osculation matrix 
M(S) and diagram of the original path tuple, as already given in (!33j) and Figure El 
are shown first, followed by the vacancy-osculation matrix M(S') and diagram of 
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the corresponding path tuple for each change point where S' is the vacancy- 

osculation set which differs from S by In each M(S'), the entry is 

highlighted, and in each path diagram the sections of paths which differ from the 
original are shown in red. In Table [21 the change points are listed together with 
the diagonal differences A(S,S') = j — i, the a' and (3' for which VOS(4, 6, a', (3') 
contains 5", and the partitions A^e,^,/?'- 



1 1 1 1 0\ 

1110 10 

1 

\0 1 1 Oy 



original 



7 — r 



'i l l l o on 

1110 10 

oooioo 

\0 1 O 0/ 



r 

(4,3) 



a i i i o on 

1 1 o o 1 o 

1 

\Q 1 1 0/ 



(2,3) 



a i i i o on 

1110 10 

o o o o o o 

v 1 1 0/ 



(3,4) 



a i i i o on 

1110 10 

oooioo 

(011010/ 



(4,5) 



a 1 1 i o on 

1110 10 
1 1 
\0 1 1 0/ 



(3,5) 



'1 1 1 1 ON 
1110 10 
OOOIOO 
\0 1 1 1/ 



(4,6) 



rr 



a i i o o on 

1110 10 

oooioo 

,0 1 1 0/ 



(1,4) 



a i i i o on 
1 1 1 o o o 
oooioo 
v o 1 1 o o 0/ 



(2,5) 



a i i i o on 

1110 10 
1 1 
v 1 1 0/ 



(3,6) 



Figure 3: Change points for the running example. 





j-i 


a' 






(4,3) 


-1 


{1,2,3} 


{1,3,5} 


(3,2,1) 


(2,3), (3,4), (4,5) 


1 


{1,2,3} 


{1,4,6} 


(3,3,2) 


(3,5), (4,6) 


2 


{1,2,3,4} 


{1,4,5,6} 


(2,2,2) 


(1,4), (2,5), (3,6) 


3 


{1,2,4} 


{1,4,5} 


(4,2,2) 



Table 2: Partitions for the change points of the running example. 



A series of lemmas on the properties of change points and of vacancy-osculation 
sets which differ by a vacancy-osculation will now be presented. In these lemmas, 
B(a, b, for any € P 2 will denote the set of points of the diagonal Dj_j([a]x[6]) 
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weakly below and to the right of 

B(a,b,i,j) := ^(i + k,j + k) k E [0, min(a— i, b— 
This set is depicted using symbols o in Figure HI 



(62) 
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Figure 4: Diagram for Lemmas 3-7. 

Lemma 3. Consider a vacancy-osculation set S E VOS(a, b) and a point E 
[a}x[b]. 

• If E S, then (i,j) is a deletion point of S if and only if 
S n (B(a,b,i,j+1) U B(a,b,i+l,j)) = 0. 

• If ^ S, then is an addition point of S if and only if 
S H (B(a,b,i,j+1) U B(a,b,i + l,j)) = and E {j = V ij} where 
(H, V) E EM(a, b) is the edge matrix pair which corresponds to S. 

The sets B(a, b, i, and B(a, b, i, j+1) are depicted in Figure 0] using the symbols o 
and □ respectively. Note that the following conditions are all equivalent. 

• Sn (B(a,b,i,j + 1) U B(a,b,i + l,j)) = 0. 

• The vertex configuration at each point of B(a, b, i, j + 1) U 
B(a, b, i + 1, j) is of type 3, 4, 5 or 6. 

• A single path of P passes through each point of B(a, b, i,j + l) U (63) 
B(a,b,i + 1, j), where P E OP(a, b) corresponds to S. 

• H M ^ Vh,i+i, for each (k,l) E B(a,b,i, j)\{(b+i-j,b)}, and 
V k i ^ H k+l)h for each (k, I) E B(a,b,i, j)\{(a, a+j -i)}. 
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Also, for any (k, I) G [a] x [b], the following are equivalent. 

• H k i = Vku 

• H Kl _ x = W (64) 

• The vertex configuration at (A;, /) is of type 1, 2, 5 or 6. 

It can be seen using Lemma 3 that the points given in Figure [3] and Table [2] comprise 
all of the change points for the vacancy-osculation set (132]) . 

Proof. Let (H, V) G EM(a, b) correspond to S, and let 5" be the set which differs 
from S by 




S\{(i,j)}, (i,j)eS 

su{(ij)}, (i,j)£s. 



It thus needs to be checked whether or not S' is a vacancy-osculation set. First, 
define H' and V by (136]) . using 5". It then follows directly from (136]) that (H,V) 
and (H', V) are related by (ISTj) . which appears in the forthcoming Lemma 4. The 
positions of the edges corresponding to H k i with (k, I) G B(a, b, H k[ with (k, I) G 
B(a,b,i+1, j), V k i with (k,l) G B(a,b,i, j), and V k i with (k,l) G B(a,b,i, j + 1) are 
shown in Figure H] using the symbols >, <, A and y respectively. It now follows that 
S' G VOS(a,6) if and only if (H',V) G EM(o,6). The first four conditions on the 
RHS of (FTBl) . which define EM(a, b), are automatically satisfied due to the definition 
of (H', V) via (1361) . so it is only the last condition, which now reads 

Ki-i + Ki = vu,i + K , (65) 

for each (k, I) G [a] x [b], that needs to be checked here. Since (H, V) G EM(a, b), 

+ y ki = V k -u + H kl (66) 

is satisfied for all (k, I) G [a] x [&]. The condition (I6"5*]) will now be examined for five 
separate cases: 

(1) (k,l)eB(a,b,i+l,j+l) 

(2) (k, I) G B(a, b, + 

(3) (k,l)EB(a,b,i+l,j) 

(4) (k,l) = (i,j) 

(5) All other (k, I) G [a] x [6] . 
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In Case (1), (J67D gives H' k ^ = l-H kM , V k ' t = l-V kU VU,i = and 
H' kl = 1 — Hki- It thus follows from fl66l) that in this case fl65l) is satisfied for all 
S G VOS(a,6). 

In Case (2), dSZD gives H' k ^ = 1-H k ^, V k \ = 1-V kl , = V k _ hh and H' kl = 

H k i. By examining the six possibilities for (H k j_i, V k i, V k -i,u H k i) in (166]) . it is found 
that in this case fl65|) is satisfied if and only if (H kt i-i, V k i, V k -i,u H k i) 7^ (1,1,1,1) 
or (0,0,0,0), or equivalently (k, I) S. Thus, Case (2) leads to the condition 
SnB(a,b,i,j+l) = 0. 

Case (3) is analogous to Case (2), and leads to the condition S fl B(a, b, = 0. 

In Case (4), flST} gives H'^ = H^, V> 3 = 1-% VUj = and H' tJ = l-H iy 
It follows that in this case fl^BT) is satisfied if and only if Vy, V^ij, H v ) ^ 

(1,0,0,1) or (0,1,1,0) (i.e., the vertex configuration at is not of type 3 or 4 
in f|TT]) ). which leads to the condition Hij = Vij if (i, j) ^ S. 

In Case (5), (JB7D gives B' kl _ x = H k ^, = V kh Vl_ u = V k ^i and H' kl = H kh so it 
follows from fl66l) that fl65l) is satisfied for all S G VOS(a, b). 

Overall then, the set S' is in VOS(a,6) if and only if the conditions arising from 
Cases 2 to 4 are satisfied, which gives the conclusions of Lemma 3. □ 

Lemma 4. Consider vacancy-osculation sets S, S' G VOS(a, b), with corresponding 
edge matrix pairs (H, V), (H', V) G EM(a, b), and a point G [a] x [b]. Then S 
and S' differ by the vacancy-osculation (i,j) if and only if 

\ H kl-H' kl \ = 6(fc,2)eB( a ,6,i,j)uB(a,&,m,j)> f or each (M) e H x [0,&], 

(67) 

and \V k i-V k \\ = 6( k ,i)eB( a ,b,i,j)\jB(a,b,ijH), /or each (k, I) G [0, a] x [b). 

The positions of the edges corresponding to H k i or H kl with (k, I) G B(a, b, i,j), H k[ 
or H' kl with (k, I) G B(a,b,i+1, j), V k i or V kl with (k, I) G B(a,b,i, j), and V k i or V kl 
with (k,l) G B(a, b, i, j + l) are shown in Figure 0] using the symbols >, <, a and v 
respectively. 

Proof. If S and S' differ by the vacancy-osculation (i, j), then flBTl) follows directly 
from fl36l) . Conversely, if fl67|) is satisfied, then, by considering the same five cases 
for (k, I) G [a] x [b] as in the proof of Lemma 3, it is found, using arguments similar 
to those used in that proof, that in Case (4) (k, I) = (i,j) is an element of exactly 
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one of S or S', while in each of the other four cases (k, I) is an element of both or 
neither of S and S", so that S and S' differ by □ 



Lemma 5. Consider a vacancy-osculation set S G VOS(a, b), with corresponding 
edge matrix pair (H, V) G EM(a, 6), and a point G [a] x [6]. T/jen i/te following 
are all equivalent. 

• is a change point of S. 

• S fl (B(a, b, + U -B(a, b, j)) = and there exists (k, I) G B(a, b, i,j) 
such that Hm = Vjy . 

• S n (-B(a,6,z,j + 1) U 5(0,6,2 + 1,^)) = and # H = 14/ /or eac/i (A;,/) G 
B(a,b,i,j). 



• 






1 













= 1 




or 




1 













1 





(6£ 



for each (k,l) G B(a,b,i, j)\{(a,a+j —i), (b+i—j,b)} 



and 



^a, a-'rj—i 




1 













or 

















if j — i < b — a . 



1 



1 



• 1 or 



.0 , if j-i = b-a. 



> a 6 



(69) 



b,b+i—j 



1 bH-j,b 







1 























, i/ j-i > b-a. 



Note that (|69|) can be regarded as the boundary case of (I68I) for 

{(a,a+j — i), j—i < b—a 
(a, 6), j — i = b — a (70) 
(b+i-j,b), j-i > b-a. 
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Proof. It will first be shown that the second condition implies the third. Thus, let 
the second condition be satisfied. If (k+1, Z+l) G B(a, b, (i.e., k ^ a and I ^ b), 
then, using the first diagram of Figure [5] to visualize the positions of the relevant 
edge matrix entries, V^i+i = 1 — H k i (since (fc, Z + l) ^ S) and H k+ ij = 1— (since 
(k + 1, 1) £ S), so that H m ,i = V k ,i+i, and therefore (using ([64])) H k+1 j +1 = V m , m . 
By repeating this process, moving in a downward- rightward direction from (k, I), it 
follows that Hkqi = Vk'i' for all (k', V) G B(a, b, k, I). 

Similarly, if (k — l,l — l) G B(a,b,i,j) (i.e., (k,l) ^ (hj)), then, using the second 
diagram of Figure [5] to visualize the positions of the relevant edge matrix entries, 
H k ^i = Vjshl.j (using = l-Vj^^j (since (/c-l,Z) ^ 5) and Vfc_i,{_i = 

1 — Hbj-i (since (k,l — 1) ^ S 1 ), and therefore Hk-i,i-i = Vk-\,i-\- By repeating this 
process, moving in a upward-leftward direction from (k, Z), it follows that H k q/ = Vkn 1 
for all (k',V) G S(a,6,i, j)\-B(o,6,*:+l,i + l)- 

z i+i 



k+1 ■ 











'Vkl 


v k,l+l 




H k+l,l 


H 'k+1, (+1 



fc-1 ■ 



l- 


-1 






v k-l,l-l 


•v k -i,i 




H k ,l-1 


H 

■Vkl 



Figure 5: Diagrams for the proof of Lemma 5. 
Overall then, it follows that Hk>v = Vk'V f° r & h (k', I') G B(a, b, i,j), as required. 

Having now shown that the second condition implies the third, the equivalence 
between the second and third conditions follows immediately, while the equivalence 
of these conditions with the first condition follows from Lemma 3. 



Finally, it can be seen that the third and fourth conditions are equivalent, using the 
equivalence between the first and fourth conditions of flo^]) . □ 

It follows immediately from Lemma 5 that if (i,j) is a change point of S G VOS(a, b), 
then all elements of B(a,b,i+l,j+l) are change points of S as well. 

It also follows from Lemmas 4 and 5 that if S, S' G VOS(a, b) differ by the vacancy- 
osculation (i,j), then whichever of the two alternatives occurs on any RHS of (l68j) 
or the RHS of fl69|) for the edge matrix pair corresponding to S, the other alternative 
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occurs on that RHS for the edge matrix pair corresponding to S'. 

Lemma 6. Consider vacancy-osculation sets S G VOS(a, b, a, /3) and S' G 
VOS(a, b, a', [3'). If S and S' differ by a vacancy-osculation, then \ a> b, a ,f3 and \ a ,b,a',f3' 
differ by a square, and 

A(S,S') = A(A ,6,a,/3, X a ,b,a',P') ■ (71) 

For S' G VOS(4, 6, a', (3') corresponding to each change point (i,j) of the run- 
ning example S, Lemma 6 states that (3,2,2) ~ ^Afi, a ',i3' and A(S,S') = j — i = 
A((3, 2, 2), \ifi,a',i3'), which can be observed in Table [2J It can also be seen in 
Table [2] and Figure [3] that for cases of Lemma 6, it does not always follow that 

IS | — 1 5" | = lAo^c^l — \\a t b,a',f3'\- 

Proof. Let S and S' differ by the vacancy-osculation {i, j), so that A(S, S') = j—i, 
and let (H,V) G EM(a,b,a,f3) and (H',V) G EM(a, b, a', (3') correspond to S 
and S' respectively. Focussing on the edge matrix entries along the right and lower 
boundaries of the rectangle, it follows from Lemmas 4 and 5 that fl67|) and fl69|) are 
satisfied, which gives, for each k G [a] and I G [b]: 

• H kb = H kb , \V a i — V al \ = ^{a^'-^o+j-i+i} and 

(V a ,c*H, K.oy-iu, Vl^, KVj-^fl) = C 1 ' °' °' X ) OT (°' X ' X > °)' if 3 ~ l < b ~ a - 

• \H kb -H' kb \ = 5 ak , \V a i-V^\ = 8 bl and 

(H ab , V ab , H' ab , K'J = (1, 1, 0, 0) or (0, 0, 1, 1), if j-i = b-a. 

• \Hkb~ H kb \ = 8k£{bi4-j,bH-j+l}, Val — Val an d 

(Hbu-jfi, HbM-j+i^ H^ t, H^^ b) = (1, 0, 0, 1) or (0, 1, 1, 0), if j-i = b-a. 

This gives six separate cases, each of which leads to the conclusions of Lemma 6. The 
details of the case j-i < b-a with {V a>aHA , K.^^-i, KiW* = (*> °> °> x ) wil1 

now be outlined, the other cases being reasonably similar. Setting a = {ot\, . . . , a r }, 
a' = {a[, . . . ,a' r ,}, [3 = {Pi, . . . , /3 r } and (3' = {f3[, . . . , f3' r ,}, with the elements of 
each set labeled in ascending order, it can be seen for the case under consideration 
that r = r' , a = a', and that there exists m G [r] such that (3 m = a+j-i, (3' m = a+j-i+\ 
and (3[ = Pi, for all I G [r]\{m}. Now setting /i = {b—Pi, . . . , b~P r \ a—oti, . . . , a—a r ) 
and // = {b — P[, . . . , b — P' r \ a — a[, . . . , a — a' r ), it can be seen using fl39l) that /i m = 
m+b—Pm = m—a+b+i—j = fi' m +l and fik = n' k , for all k G P\{m}. Therefore, Y(fi) is 
the disjoint union of F(/i') and {(m, m—a+b+i—j)}. Finally, since X a ,b,a,f3 = H X [^]V 
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and X a ,b,a',p' — H x [b]\^' from (joTl) . it follows that Y(X a ^ ta r^r) is the disjoint union 
of Y(X aAct)l3 ) and {(a - m + 1, a - i + j - m + 1)}, so that A a ,& ja ,/3 ~ K,b,a',p' and 
A(A aAQj/3 , A aj6iQ / i/3 /) = j-i = A(S, S'), as required. □ 

Lemma 7. Consider a vacancy-osculation set S G VOS(a, b, a, 0) and a point 
G [ajx^S. Then is an addition point of S if and only if SC\(B(a,b,i,j+ 
1) U B(a, b, j)) = and j — i is a change diagonal of \ a ,b,a,i3- 

Proof. If (i, j) is an addition point of S, then it follows from Lemma 3 or 5 that 
S fl (B(a, b, i, j + 1) U B(a,b,i + 1, j)) = 0, and it follows from Lemma 6 that j — i 
is a change diagonal of X a ,b,a,/3 (since being an addition point of S means 

that there exists S' G VOS(a, 6, <*',/?') with S ~ 5' and 5' = 5 U for 
some (a',/?') G BP(a, 6), and Lemma 6 then implies that X a ,b, a ,/3 ~ X a ,b,a>,p> and 
A(A 0)6 ,a,/j, A a)6iQ / ij3 ') = A (5, 5") = j-i). 

Conversely, if j—i is a change diagonal of \ a ,b,a,p and 5 fl (-B(a, b, i, j+l) UB(a, 6, z+ 
1, j)) = 0, then it follows from Lemma 5 that (i, j) is an addition point of S provided 
that H ki = Vki for some (k,l) G B(a,b,i,j), where (H,V) G EM(a, b) corresponds 
to S. It will be shown that H k i = V k i is satisfied for (k,l) given by flTUj) . First, it 
follows from (|55j) and (!58l) that, for any d G Z, 



the first terms on each RHS of fl72l) and (1731 being equal by elementary geometry 
of the rectangle. 

Next, it can be shown using (|46p and f!73|) that only the possibilities 



Pd(Kb,a,/3) = Rd([a)x[b]) 



\{i G [r] | $ < a+d}|, d < b-a 
\{i G [r] | a.i < b-d}\, d > b-a. 



(72) 



This can be rewritten as 




(73) 



• (K.atf-i, K.stfw) = (1, 0) or (0, 1), if j-i<b-a 

• (#„ 6 , Kb) = (M) or (0,0), if j-i = b-a 

• (#^,6, i^ w _ i+ i i6 ) = (1, 0) or (0, 1), if j-z > fo-a , 



(74) 
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can occur for the edge matrix entries at the two edges adjacent to the diago- 
nal Dj_j([a] x [b]) on the right or lower boundary. The details of the case —a < 
j — i< min(6 — a, 0) will now be outlined, all other cases being similar. For this 
case, (J73]) gives pj^-i(\ a ,b,a,p) = a+j -i-l-J2'^r~ 1 V ah Pj-i{K,b, a ,p) = a+j—i- 
Y^V V ai and Pj-4u(\ a ,b,a,p) = a + j - i + 1 - E^o* v ai- Therefore, {pj~i{X a ,b,a,p) ~ 

Pj-ir-l[X a ,b,a,p), Pj~4hl[X a ,b,a,p) — Pj-i\X a ,b,a,p)) = (1 — Va,a+j-4, 1 — ^a,o4^-4fl) ■ But, Us- 
ing (USD, this must be (1,0) or (0,1), implying that (K,otH, K^j-i+i ) = (1,0) or 
(0, 1), as required. 

It can now be seen that, as required, H ki = V k i for (k, I) given by (170|) . this following 
from (!74j) and (a,a+j—i+l) S (since (a,a+j—i+l) G B(a,b,i, j+1)) if j— i < b—a, 
from (!74|) alone if j— z = 6 — a, and from (1711) and (b + i — j + l,b) £ S (since 
(b+i-j+l,b) G B(a,b,i+l,j)) ifj-i > b-a. □ 

9. Vacancy-Osculation Set Progressions 

In this section, it will be shown that the elements of any vacancy-osculation set S 
can be ordered in such a way that, for each k G [\S\], the first k elements themselves 
constitute a vacancy-osculation set. 

For a finite set T, let an ordering of T be any permutation, represented as a tuple, 
of all the elements of T. If T is a subset of Z 2 , let a canonical ordering of T be any 
ordering ((ix,ji), ■ ■ ■ , (i\T\,j\T\)) for which if k < I, with k, I G [\T\], then z fc < ^ or 
j k < ji- Furthermore, let the lexicographic ordering of T be the unique canonical 
ordering ji), . . . , (i\T\,j\T\)) which satisfies i k < i m , or i k = i m and j k < j m , 
for each k G [|T| — 1]. (For example, in (I32I) the points of 5* are written in such an 
order.) 

For a vacancy-osculation set S G VOS(a, b, I), define a progression for 5 to be any 
ordering s = (sj, . . . , sj) of S 1 which satisfies {si, . . . , s k } G VOS(a, 6) for each k G 
[0, 1]. A progression s for S therefore satisfies 0~ {si} ~ {s\, s 2 } ~ . . . ~ {si, . . . , si}, 
with s^. being a deletion point of {si, . . . , s k } and an addition point of {si, . . . , 
for each k G [Z]. 

Lemma 8. v4ny canonical ordering of a vacancy-osculation set is a progression for 
that set. 
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Proof. Let s = (si, . . . ,s{) = ((«i, Ji), • • • , be a canonical ordering of S G 

VOS(a, b, I), and consider {si, . . . , s m } for any m G [/]. Since s is canonical, i*. < i m 
or Jfc < jm for each A; G [m— 1], from which it follows, due to fl62l) . that {sj, . . . , s m } fl 
(B(a,b,i m ,j m +1) U B(a,6,i m +1, = 0- Therefore, if {si, . . . , s m } G VOS(a,6), 
then, by Lemma 3, s m is a deletion point of {s\, . . . , s m }, and so {si, . . . , s m _i} G 
VOS(a, b). Since {si, . . . , s{\ = S G VOS(a, b), it follows by repeated application of 
this result starting with m = I that {si, . . . , s m } G VOS(a, b) for each m G [0, 1], so 
that s is a progression for S. □ 

Corollary 9. There is at least one progression for any vacancy-osculation set. 

Proof. Since the lexicographic ordering of a vacancy-osculation set is a canonical 
ordering, it follows from Lemma 8 that it is a progression for that set. □ 

An example of a canonical, but non-lexicographic, ordering of the vacancy-osculation 
set ( 1321) of the running example, and hence a progression for that set, is 

s = ((1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (1, 4), (3, 4), (2, 5), (4, 2), (4, 3)) . (75) 
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Figure 6: A progression for the running example. 



In Figure [6l each Sk for this example is shown together with M({s\, . . . , Sk}) and 
the diagram of the path tuple corresponding to {si, . . . , Sk}, for k G [0, 11]. In each 
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matrix, the Sk entry is highlighted, and in each path diagram the sections of paths 
which differ from the previous diagram are shown in red. 

Note that the converse of Lemma 8 is false. For example, the ordering obtained 
from 0751) by interchanging (1,4) and (2,5) is no longer canonical, but is still a 
progression for the vacancy-osculation set fl32|) . 



10. Ordering Pairs 

In this section, it will be shown that progressions can be systematically obtained 
for any vacancy-osculation set by using a function on P 2 and an order on Z which 
satisfy certain properties. 

A total strict order on a set T is a binary relation -< on T for which: 

(1) Exactly one of t\ -< t2, h = t 2 or t 2 -< ti holds for each tx, ti G T. 

(76) 

(2) If t\,t 2 ,t 3 G T satisfy ii -< t 2 and t 2 -< t 3 , then tx -< t 3 . 

Of primary interest here will be total strict orders on Z. The standard less-than 
relation < is one example of a total strict order on Z, while another example is that 
defined, for each z,z' G Z, by 

z-<z' if and only if \z-2\>\z'-2\ or z-2 = 2-z'<0, (77) 

which gives ...^-1^5^0^4^1^3^2. 

Let an ordering pair (L, -<) be any pair comprised of a function L : P 2 —>■ P and a 
total strict order -< on Z, which satisfy the property, for G P 2 , that 

If L(i,j) < L(i',f), or L{i,j) = L(i',f) and j-i^j'-i', 
then i < i' or j < j' . 

The function L will be referred to as a level function, and L(i,j) will be referred to 
as the level of G P 2 . The full motivation for the use in (1781 of a total strict 
order on the diagonals of P 2 (i.e., the use of j— % -< j'—i' rather than a more general 
condition for the case L(i,j) = L(i',j')) will become apparent in Section 13. 

It follows, for an ordering pair (L, -<), and (i',f) £ P 2 , that 

If L(i,j)=L(i',f) and j-i=f-i', then (i, j) = (i', f) . (79) 
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In other words, each point G P 2 is uniquely determined by its level L(i,j) 

and content j—i. Property (1791) can be derived as follows. Let L(i,j) = L(i',j') 
and j — i = j' — i', and consider, for example, the point (i,j'). By applying (175]) to 
(i, f) and {i, j), and to {i, f) and (i',j'), it can be seen that none of the possibilities 
L(i,f) < L(i,j) = L(i',f), L(i,j) = L(i',f) < L(i,f), L(i,f) = L(i,j) = L(i',f) 
and j'—i -< j—i = j'—i', or L(i,j') = L(i,j) = L(i',j') and j—i = j'—i' -< j'—i, can 
occur, leaving only j' — i = j—i = j' — i', and therefore = {i' ,j'). 

It follows from f[75|) and (1791 that for an ordering pair (L, -<) and a vacancy-oscu- 
lation set S G VOS(a, b, I), there exists a unique canonical ordering s = ■ ■ ■ , 
of S which satisfies 

L(ik,jk) < L{i k+l ,j k+ i), or L{i k ,j k ) = L{i k+ i,j k+l ) and j k -i k -< j k+ i-i k+ \ , (80) 

for each k G [I — 1] . (Alternatively, a total strict order -< on P 2 can be defined by 
(hj) -< (i',f) if and only if L(i,j) < L{i',j'), or L{i,j) = L{i',j') and j-i -< 
j' — i', and s = ((i\,ji), ■ ■ ■ , is then the unique ordering of 5" which satisfies 

(ik,jk) ~< (ik+i,jk+i) for each k G [I — 1].) By Lemma 8, this s is a progression for S. 
Accordingly, s will be referred to as the progression for S induced by the ordering 
pair (L, -<). 

An example of an ordering pair is (L, <), where the level function is 

L(i,j) = i, for each (i,j) G P 2 , (81) 

and the total strict order is the standard less-than relation, since it can be seen 
immediately that these satisfy (1751) . The progression for a vacancy-osculation set 
induced by (L, <) is the lexicographic ordering of the set, and accordingly (L, <) 
will be referred to as the lexicographic ordering pair. Similarly, another example of 
an ordering pair is (L, >), where the level function is 

L{i, 3 )= 3 , for each G P 2 , (82) 

and the total strict order is the standard greater-than relation. This will be referred 
to as the antilexicographic ordering pair. 

A third example of an ordering pair is (L, -<), where the level function is 

L(i,j) = max(i, j-2), for each G P 2 , (83) 
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and the total strict order is given by the example (1771) . In checking that these 
satisfy (1751) . it may be useful to visualize ([83]) as 



L(2,l) 

V : 



L(l,2) 
£,(2,2) 



/l 1 1 2 3 

2 2 2 2 3 

3 3 3 3 3 

V 



34) 



7 



It can be seen that the progression for the running example (132|) induced by this 
ordering pair is that of (175]) . 



11. Oscillating Tableaux 



In this section, oscillating tableaux will be introduced, in preparation for the con- 
sideration in subsequent sections of their relationship with vacancy-osculation sets. 

For a partition A and a nonnegative integer I, an oscillating tableau of shape A and 
length I is a sequence of l+l partitions starting with 0, ending with A, and in which 
successive partitions differ by a square. Let OT(A, I) denote the set of all such 
oscillating tableaux, 



OT(A,/) := 

{v = (?7o = 0,?7i, • • • ,Vi-i,Vi = ^) e Par 



m 



Vk-i 



7] k for each k G [I] j . 
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Oscillating tableaux are sometimes called up-down tableaux, and were first intro- 
duced in [6]. Oscillating tableaux, or certain generalizations thereof, are studied in 
the context of Robinson-Schensted-Knuth-type correspondences in [61 [131 HH ESI El 
1221 EH EH SSI US EDI EH ESI ED] , and in the context of nonintersecting lattice paths 
in [31 [32]. They are also discussed in, for example, [51], [5H Exercise 7.24] and [551 
Sec. 9]. 

Let a deletion of an oscillating tableau rj G OT(A, I) be a value G [/] for which 
1%! — l^fc-il - 1- Denoting the number of deletions in 77 as T)(rj), it can be seen that 



V{ri) = (Z — I A I )/2 , for each 77 G OT(A,/) , 
and that OT(A,Z) = unless (/ — |A|)/2 is a nonnegative integer. 
For an oscillating tableau 77 G OT(A, I), define 

Q(rj) k := A(r? fc _i,77 fc ), for each fc G [Z], 



(86) 
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and define the profile of r\ to be the sequence of these diagonal differences, 
n( V ) := (n(rj) u n(rj) 2 ,... : n(rj) l ) = (A(77o,77i),A(77 1 ,7 te ),...,A(77,_ 1 ,77,)). (88) 

In other words, fl(rj) is the sequence of contents of the squares by which the Young 
diagrams of successive partitions of i] differ. 

It can be seen that an oscillating tableau r] is uniquely determined by its profile £l(r)), 
since rj is always and, as indicated in Section 6, for fixed \x G Par, each // G Par 
with is uniquely determined by A(/i, /i'). 

It can also be seen that, for any r\ G OT(A, /), 

For each d G [-£(X) + 1, Ai-1], there exists fc G [/] with Q(i]) k = d. (89) 

It can be deduced straightforwardly that, for any rj G OT(A, I) and k G [/], 

• If fi(r])fc < 0, then there exists k' G [A; — 1] with Q(r))k' = ^(^)fc + l- 

• If {l(r])k > 0, then there exists k' G [k — l] with Q(r))k' = £l(r]) k — l. 

(90) 

• If {l(ri) k < —£(X), then there exists k' G [/c + l,/] with Q(r))k> = fi(^) fc + l. 

• If Q(i])k > Ai, then there exists k' G [A;+1,Z] with 0(77)^/ = 0(77)^ — 1 . 

For an oscillating tableau G OT(A, /) and a total strict order -< on Z, define the 
set of ascents ofrj with respect to -< as 

A(^rj) := {ke[l-l]\n(r,) k ^n(r,) m }, (91) 

and the number of ascents ofrj with respect to -< as 

A^,v) ■= 1^,77)1 . (92) 

Also define the distribution function on OP(A,Z) of the number of ascents with 
respect to -< as 

Af(\,l, -<,i) := \{r] G OT(A,/) I A{-<,rj) = i}\, for each % G [0, 1-1} . (93) 

An example of an oscillating tableau r\ of shape (3, 2, 2) and length 11 is 
77 = 

(0, (1), (2), (3), (3,1), (3,2), (3,3), (4,3), (4,2), (3,2), (3,2,1), (3,2,2)). (94) 
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The Young diagrams for each partition of 77, and the terms of the profile of 77, are 
given in Table [31 It can be seen that 8 and 9 are the deletions of 77, and that for the 
total strict order on Z given in (1771) . 

AH, 77) = {1,2, 4, 5, 6, 8, 10}. (95) 



10 



11 



a) 
□ 



(2) 

m 



(3) 

xn 



(3,1) 

F 3 



(3,2) 



(3,3) 



(4,3) 



(4,2) 



(3,2) 



(3,2,1) 



(3,2,2) 



XI 



Table 3: Example of an oscillating tableau. 

The enumeration of oscillating tableaux, and their relationship with standard Young 
tableaux, will now be discussed. For a partition A and a finite subset T of P with 
\T\ = |A|, define the corresponding set of standard Young tableaux as 

SYT(T, A) := I er • o~ is an array of entries G T with (i,j) G Y(A) 

• cry < o-j^! for each z G [^(A)], j G [A — l] (96) 

• <Jij < (Ji\-ij for each i G [£(A) — 1], j G [A^i 

|A|], define 

SYT(A) := SYT([|A|],A). (97) 



For the case T 



A standard Young tableau o G SYT(T, A) is usually shown by writing each cr^- 
within the unit square centered at (2, j) in the Young diagram of A. These entries 
thus increase along each row and down each column, and together comprise T. 
It follows immediately that |SYT(T, A)| is independent of T, this number usually 
being denoted as f x . A well-known formula for f x is the hook-length formula (see 
for example [541 Corollary 7.21.6]). An example of an element of SYT((4,2,2)) is 



a 



1 


2 


3 


7 


4 


6 




5 


8 



(98) 



It can be seen that each oscillating tableau r] of shape A with zero deletions, i.e., 
77 G OT(A, |A|), can be bijectively associated with a standard Young tableau o G 
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1 1 
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1 1 


1 1 
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-1 


-2 





3 


-1 



Table 4: The oscillating tableau 77 which corresponds to a of (j98p . 

SYT(A), where, for G Y(X) and k G [|A|], Y(%) = F^) U if and 

only if &ij = k. For example, the oscillating tableau 77 G OT((4, 2, 2), 8) which 
corresponds to a of ( 1981) is shown in Table HI 

Now define the set of ascents A(o~) of any a G SYT(A) to be the set of integers k G 
[|A|— 1] for which k+1 appears in a in the same or a higher row than does k, and denote 
the number of ascents of a as A(cr) := |A(er)|. For example, A(a) = {1, 2, 5, 6} for o 
of ( 1981) . It can be seen that if a G SYT(A) corresponds to 77 G OT(A, |A|), then 
A(cx) = A(<,77), and therefore A{a) = A(<,rf). 

In [571 EH], the general case of oscillating tableaux with n deletions is considered, 
and it is shown that OT(A,Z), with / = |A|+2n, can be bijectively mapped to the 
set of pairs (M, a) in which M is a matching on a set T (i.e., a decomposition of T 
as a disjoint union of 2-element subsets), for any T C [/] with |T| = 2n = I — |A|, 
and a is a standard Young tableau in SYT([/]\T, A). The number of such sets T is 

^ ^ and, for any T, the number of matchings M is (I — |A| — 1)!! and number of 

standard Young tableaux a is f x , which gives 

|OT(A,Z)| = (|Al) a-l^l— 1)H/ A , (99) 

for (/-|A|)/2 G N. 

12. Progressions and Oscillating Tableaux 

In this section, it will be shown that any progression for a vacancy-osculation set 
can be simply and naturally associated with an oscillating tableau. 

Theorem 10. Consider a progression s = ((«i, ji), . . . , for a vacancy-oscu- 

lation set S G VOS(a, b, a, /3, 1). Then there exists an oscillating tableau r)(s) of 
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shape X a ,b,a,/3 o,nd length I with the profile 

n{v{s)) = (100) 

Furthermore, rj(s) has the same number of deletions as the number of osculations 
ofS, 

V( V (s)) = X (S) , (101) 

and if s is a canonical ordering, then, for each k G [I], k is a deletion of rj(s) if and 
only if(ik,jk) an osculation of S. 



As indicated in Section 11, any oscillating tableau is uniquely determined by its 
profile, so that (11001) defines a unique oscillating tableau rj(a) for the progression s. 
This will be referred to as the oscillating tableau associated with s. 

Proof. For each k G [0,1], denote ■ ■ ■ , (ik, jk)} as S h , and let (a k ,(3 k ) G 

BP(a, b) be the boundary point pair for which S G VOS(a, b, a k , (3 k ). Now define 
the sequence of partitions 

V = (u ,...,Ui) := (\ a ,b,a ,l3°i ■ ■ • , K,b,a l ,0 l ) ■ (102) 

Since s is a progression for S, it follows that S^" 1 ~ S k , so Lemma 6 implies that 
X^ak-i pk-i ~ \ a ,b,a k ,p k i v k-\ ~ v ki for each k G [I]. The boundary point pair 
for S° = is (a°,P°) = ([min(a, b)), [min(a, 6)]), so fl54l) gives u = \ at b, a °,f3 = 0, 
and the boundary point pair for S is (a 1 , ft 1 ) = (a,/3), so v\ = \ a ,b,a',(3 l — ^a,b,a,p- 
Therefore, = v ~ V\ ~ . . . ~ ~ V\ — X a ,b,a,/3, so v is an oscillating tableau of 
shape X a ,b,a,/3 and length I. Applying flTTT) and (JS7]1 to S k ~ l and 5* fc for each k G [/] 
gives VL{y) = (ji—ii, ■ ■ ■ — which matches the RHS of (11001) . implying, since 
any oscillating tableau is uniquely determined by its profile, that r](s) = v. The 
result (110 II) now follows by applying (!60|) and (!86|) to S and rj(s). 

Finally, consider a fixed k G [/]. Then applying (I6D1) to S ,fe and S^ -1 gives /c = 
|?7(s) fe | + 2x(S k ) = l + \r](s)k-i\ + 2x(S k ~ 1 ), so that is a deletion of r](s), i.e., 
|^(s)fc| = \v( s )k-i\— 1, if and only if x(>S' fc ) — x(*S' A:_1 )+l- Now it follows from Lemma 4 
that a point of S^ -1 is an osculation of S k ~ l and a vacancy of or a vacancy of 
S^ 1 and an osculation of S k , if and only if the point is in 5 ,fc_1 fl B(a,b,ik,jk), 
i.e., (X(5 fc - 1 ) n X(^ fc )) U (N(S k ~ 1 ) n X(S fc )) = S^ 1 n B(a,b,i k ,j k )- However, if s 
is a canonical ordering, then S^ 1 fl B(a,b,ik, jk) = 0- It can now be seen that 
for this case the following statements are all equivalent: k is a deletion of r](s), 
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x (S k ) = x (S k ~ 1 ) + l : X(S k ) is the disjoint union of X^ 1 ) and {s k }, s k G X(S k ), 
s k G X(S k ) n X(S fc+1 ) n . . . n X(S l ), s k G X(S) (where s k = (i k ,j k )). □ 

For the example of a progression s = {(h,ji), ■ ■ ■ > in)} of (1751) and Figure El 
(ji — ii, ■ ■ ■ ,ju~hi) — (0, 1, 2, —1, 0, 1, 3, 1, 3, —2, —1). This matches the profile for 
the example of an oscillating tableau of (|94"|) and Table [3J which is therefore the 
oscillating tableau associated with s. Also, by Theorem 10, since s is a canonical 
ordering, the deletions 8 and 9 of the oscillating tableau correspond to the osculations 
(^8)Js) = (3,4) and (^9,^9) = (2,5) of the vacancy-osculation set. 



13. Generalized Oscillating Tableaux 

It was seen in the previous section that a vacancy-osculation set can be associated, 
via a progression, with an oscillating tableau. However a progression is not, in 
general, uniquely determined by its associated oscillating tableau. In this section, a 
generalization of oscillating tableaux will be introduced, and it will be shown that, 
via the progression induced by an ordering pair, a vacancy-osculation set can then 
be uniquely associated with such a generalized oscillating tableau. 

For any subset T of P, total strict order -< on Z, partition A and nonnegative 
integer /, define the corresponding set of generalized oscillating tableaux as 

GOT(T, A, I) := {(t, r?) = (fa, ...,t{), 77) eT l x OT(A, l)\t k < t m , 

or t k = t k+ i and k G A(~<, rj), for each k G [/ — 1 

= {(t, 77) GT'x OT(A, /) t k < tk+i, or t k = t m and 
fl(r)) k -< fl(rj) k+ i, for each k G [/ — 1 

where in the second line, the definition (l9Tj) of A(-<,rj) has simply been used. 

It follows that, for finite T, 



(103) 



|GOT(T,^,A,Z)| = E 

r?eOT(A,0 



I 



E AA(A,/,^,z) 



(104) 



,_f] \ ^ 



where definitions (}9"2"]) and fl93|) have been used. 
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Using the example (|94"j) of an oscillating tableau n, and the example (jTTjl of a total 
strict order -< on Z, for which v4(^,n) is given by fl95|) . it can be seen that an 
example of a generalized oscillating tableau in G0T([4], -<, (3, 2, 2), 11) is 



(t,rj) = ((1,1,1,2,2,2,2,3,3,4,4), 

(0, (1), (2), (3), (3, 1), (3, 2), (3, 3), (4, 3), (4, 2), (3, 2), (3, 2, 1), (3, 2, 2))). 



Theorem 11. Consider a, b G P, (a, (3) G BP (a, 6), Z G N, and an ordering pair 
(L, -<). For eaca vacancy-osculation set S G VOS(a, 6, a, /5, /), Zet s fre tae progres- 
sion for S induced by (L, -<) and n(s) fre tae oscillating tableau associated with s, 
and define 



oscillating tableaux GOT : (L([a]x[6]), -<, X a ,b,a,p, 0; where L([a\x[b\) := {L(i, j)|(i, j) G 



Proof. The definition of L([a]x[6]) immediately gives (L(si), . . . , L(si)) G L([a]x[6])', 
while Theorem 10 gives <q{s) G OT(A a , 6ia ^, Z). Also, (150} . and fTTOOj) give L(s fc ) < 
L(sfc + i), or L(sfc) = L(sfc+i) and G A(-<,rj(s)), for each fc G [I — 1]. Therefore, 
by (HMD, Q{lM S ) e GOT(L([a]x[6])^,A aM/3 ,l) for each 5 G VOS(a, 6, a, /3, 1). 

Now consider S", S' G VOS(a, 6, a, (3, 1) for which 6(l i ^)(S') = Q(l,^)(S'). It follows 
that L(s k ) = L{s' k ) for each k G [Z], where s and s' are the progressions for S and 5" 
induced by (L, -<). The associated oscillating tableaux are also equal, n(s) = n(s'), 
so setting s k = (i k ,jk) and 4 = {i' k ,j' k ), (11001) gives j fc -4 = j*-^ for each fc G [/]. 
Therefore, by f!79|) . (ik,jk) = (*Jfe 3 J&) f° r eac h k G [Z], giving s — s' and 5 = S", and 
implying that 0(l,-<) is injective. □ 

For an ordering pair (L, -<), ©(l,^) : VOS(a, 6, a, /3, Z) — > GOT(L([a]x[6]), -<, X a ,b,a,/3, 1) 
as defined by (11061) will be referred to as the mapping induced by (L, -<), and, for 
any vacancy-osculation set S G VOS(a, b, a, /3, Z), 6(l > ^)(5') will be referred to as tae 
generalized oscillating tableau associated with S via (L, -<). 

For the vacancy-osculation set S of (1521) and the ordering pair (L, -<) given by (177)) 
and (1531) . it has already been seen that the progression for S induced by (L, -<) is s 
of (175|) . and that the oscillating tableau associated with s is that of (194|) . It can now 
be seen that the generalized oscillating tableau associated with S via (L, -<) is that 




[a]x[6]}. 



of PS}. 
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14. Bijections Between Vacancy-Osculation Sets and Generalized 
Oscillating Tableaux 

It was seen in the previous section that any ordering pair (L, -<) leads to an in- 
jective mapping from any set VOS(a, b, a, (3, 1) of vacancy-osculation sets to the set 
GOT(L([a] x [&]), -<, X a ,b,a,/3, of generalized oscillating tableaux, and it will be seen 
in this section that there exist certain ordering pairs for which this mapping is always 
bijective. 

Lemma 12. Consider a,b G P, (a, (3) G BP(a,6) ; / G N, an ordering pair (L, -<), 
and (t,rj) G GOT(L([a] x [b]), -<, X a ,b,a,Pi 0- tf> f or eac h k e there exists Sk = 
{ik-ijk) £ [a]x [b] such that L(sk) = tk and jk~ik — ^{v)k, then is unique, the 
set S = {si,...,S[} is in VOS(a, b, a, (3, I), and (t,r)) is the generalized oscillating 
tableau associated with S via (L, -<). 

Proof. For each k G [I], let there exist Sk = (ik,jk) £ N x [b] with L(sk) = tk and 
jk~ h = Q(v)k, Then Sk is unique due to ( 1791) . Also, since {t,rj) G GOT(L([a]x[fe]), -<, 
A a ,6,a,^) 0> ^ follows that tfc < tk', or = ty and -< ^(77)^, for each k, k' G [I] 

with < fc', and hence that L(sk) < L(sy), or L(sk) = L(sy) and jk—ik -< jy—iy- 
Therefore, by (!78|) . < zV or j*. < jV, from which it follows, due to (|62|) . that for 
each m G [0, / — 1], 

{Si, . . . ,S m } fl (5(0,6,^, + U 5(0,6,1,^1 + 1, jmfl)) = 0. (107) 

Now consider any m G [0,/ — 1], and assume that {si,...,s m } £ VOS(a, 6, a m , f3 m ) 
and Aa^^m^ = r] m for some (a m ,/? m ) G BP(a, 6). It will be shown that, in 
this case, {si, . . . , Sm+i} G VOS(a, b, a" 1 ^ 1 , (3 m¥1 ) and A a fc a ™+i j( gm+i = 77^1 for some 
(a nb¥1 , (3 ntH ) G BP (a, b). Since 7? m ~ 77^1 and = jWi~Wi> jWi~Wi is a 

change diagonal of A^^m^m . Using this and (11071) . it follows by Lemma 7 that s^i is 
an addition point of {s\, . . . , s m }. Therefore, {si, . . . , s^} G VOS(a, 6, a" 14 " 1 , /J™ 44 ) 
for some (a"^ 1 , /J"^ 1 ) G BP (a, 6), and Lemma 6 implies that \ a ,b,a m ,/3 m ~ A a biQ ,™fi j/3 ™+i 
and jm+i— imH = ^■{Xa,b,a m ,fi m ,^a,b,a m ^ 1 ,p m * 1 )i so that (since any partition // with 
Aa,6,a™/3™ ~ /i is uniquely determined by A(\ aAa m^m, //)) A^^i^i = 77^1. 

Since G VOS(a, 6, [min(a, 6)], [min(a, 6)]) and, from ([MD, A^^^^in^)] = 
= 770, it follows by repeated application of the result from the previous paragraph 
starting with m = that, for each m G [0, 1], {s\, . . . , s m } G VOS(a, b, a m , f3 m , m) 
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for some (a m ,/3 m ) G BP(a, b) with \ a ,b,a m ,i3 m = Vm- Taking m = I gives S = 
{si, . . . , si} G VOS(a, b, a, j3, 1). It can also be seen that (si, . . . , s{) is the progression 
for 5 induced by (L, -<), that 77 is the oscillating tableau associated with s, and that 
therefore (t,rj) is the generalized oscillating tableau associated with S via (L, -<).□ 



For any g, i, j G Z, define 



L q (i,j) 



fmax(i+g, j), q < 
(max(i,j- 5), g > 



(108) 



or, more compactly, L q (i,j) = max(min(i, i + q), min(j, j — q). Usually, L q will be 
regarded as a level function from P 2 to P for some fixed q, and in this case it can be 
visualized as 



£(1,2) 
£(2,1) £(2,2) 

V : 




q < 



(109) 



q > 0. 



It will also be seen that for the case of osculating paths in an a by b rectangle, q is 
taken to be b— a. 



It follows from (1751) that for q G Z and a g-dependent total strict order -< q on Z, 
(L g , -< g ) is an ordering pair if and only if ~< q satisfies the property that, for z, z' G Z, 

If z < z < q or z > z > q, then z -< q z' , (HO) 

i.e., . . . ^ q q — 2 ^ q q — 1 -< q q and . . . ~< q q+2 -< q q+1 ^ q q. Any total strict 
order -< q on Z which satisfies ( 11101) will be referred to as a q-order. 

A specific example of a g-order is that defined, for any z, z' G Z, by 

z -< q z' if and only if \z — q\>\z' — q\ or z — q = q — z' < , (HI) 

i.e., ... -< g q-2 -< q q+2 -< q q-1 -< q q+1 -< q q. 

Any ordering pair (L q , -< q ) in which the level function L q is given by H108[) (and -< q 
is therefore a g-order) will be referred to as a complete ordering pair. It can be seen 
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that the example of an ordering pair (L, -<) given by (I77j) and flBBl is the complete 
ordering pair (L 2 , -< 2 ) with -< 2 given by (11 111) . 

Now define, for any q,u,z G Z, 

'(u — z,u), q < min(0, z) 



G q (u,z) 



'1121 



u — q,u — q+z) , 2: < g < 
u + q — z,u+q) , 0<q<z 
{u,u + z) , g > max(0, z) 



It can be checked straightforwardly that for q, i,j, u,z G Z, 

(L q (i,j),j-i) = (u,z) if and only if (i,j) = 2) . (113) 

Finally, in preparation for Theorem 13 define, for any q G Z, T C P, total strict 
order -< on Z, A G Par, Z G N and (t, 17) G GOT(T, -K, A, Z), 

$ ff (*,tj) := {G g (ti, fi(?y)i), • • • , G^fa, fi(»7)j)} • (114) 

Theorem 13. Consider a, 6 G P, (a,/?) G BP (a, 6), and Z G N. TZzen i/ie map- 
ping 1 0(i(^,x6-„) induced by any complete ordering pair (L^, -<b-a) is a bijection 
from VOS (a, b, a, ft, I) to GOT([min(a, £>)], -<b-a, ^a,b,a,Pi 0; anc ^ ^ e inverse mapping 

IS ^b_a. 

Proof. The mapping ©(l^,^^) induced by a complete ordering pair (L^, -<im) is 
given by (11061) . By Theorem 11, it is an injective mapping from VOS(a, b, a, /3, Z) to 
GOT([min(a, &)], -<b_a, A ai fe iaij g, Z), with Lj ) _ a ([a]x[6]) = [min(a, b)} following from (11081) . 
Thus it only remains to be shown that it is a surjective mapping and that its inverse 
is $6_a. This will now be done explicitly for the case a < b, i.e., min(a, b) = a. 
The case b < a follows from the case a < b by reflection of the lattice in the main 
diagonal using the first bijection of (jSJ). 

Consider any (t, 77) G GOT([a], -<b-a, A a ,&,a,/3j 0> ana - abbreviate 0(77)^ as fife for each 
k G [Z]. By (jIUgj) . 1 < t x < . . . < t, < a, and for n,n' G [Z], 

If n < n', and Q n > -<b-a fin or fi ra = fi n /, then t n < t n > . (H5) 
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Now, for each k G [I], define Sk = (ik,jk) by 




(tk,tk + fik) , Vtk<b—a 
(t k -Qk + b-a,t k + b-a) , Q k >b-a, 



(116) 



the explicit form here following immediately from the definition (11121) of G q , with 
q = b — a > 0. It will first be shown that Sk G [a]x [b] for each G [/]. It can 
be seen straightforwardly that 1 < < a < 6 implies that ik > 1 for fi^ < 6 — a, 
Jfc > 1 for fifc > 0, ik < a and < 6. Therefore it only remains to be checked 
that ik > 1 for fifc > 6— a, and jfc > 1 for Qk < 0. If Qk > b — a, then, by applying 
the second property of (1901) fi^ — 6+a times in succession, it follows that there exist 
integers 1 < n < rii < . . . < n^ k ^ a = k such that Q Um = b — a + m for each 
m G [0,Ofc — b + a\. Since, by (1 11 01) . Qk -<b-a ^fc — 1 ^b-a ••• ^^-a fr — a, it follows 
using (11151) that t Um > m+1 for each m G [0, fi^— 6+a] and so, taking m = Qk— 6+a, 
that tfc > Ofc — 6+a+l, i.e., by (11161) . that z& > 1 as required. Similarly, if Qk < 0, 
then, by applying the first property of fl90|) — Qk times in succession, it follows that 
there exist integers 1 < n < rii < . . . < n_^ k = k such that Q nm = —m for each 
m G [0, —Qk]- Since, by (II 101) . Q k ^b-a ^fc + 1 -<b-a ■ ■ ■ ~<b-a 0, it follows using f 1 1 1 5 j) 
that t nm > m+1 for each m G [0, —Qk] and so, taking m = —Qk, that tk > —Qk + 1, 
i.e., by (11161) . that jk > 1 as required. 

Having now verified that Sk G [a] x [b], and observing from (11131) and (11161) that 
Lb-a{sk) — tk and jk~ik — Qk for each k G [/], it follows by Lemma 12 that 
G VOS(a, b, a, (3, 1) and that 9(l^,^){{si, . . . , s t }) = (t,rj), so that 
®{Lb-a,~<±-a,) * s surjective, with the inverse mapping being &b-a- 1=1 

Returning to the running example, if it were not already known that (t, if) of H 1 5 1) 
is the generalized oscillating tableau associated with the vacancy-osculation ma- 
trix S of (1321) via the complete ordering pair (L, -<) of (1771) and (183"]) . then S could 
be obtained by simply evaluating the profile for m as done in Table [3j and then 
applying $ 2 to give S = {G 2 {h, Q(v)i), • • • , G 2 (tn, ^C 7 ?)!!)}- 



Corollary 14. Consider a, b G P, (a, /3) G BP(a, 6), I 6 N, and a (b— a) -order -<b-a- 
Then 



Proof. This result follows immediately from Theorem 13, (11041) . and the bijection 





(117) 



between OP(a, b, a, j3, 1) and VOS(a, b, a, (3, 1). 



□ 
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It was seen in Theorem 13 that there is a bijection between each set of vacancy- 
osculation sets VOS(a, b, a, f3, 1) and a set of generalized oscillating tableaux, and 
it will now be seen in Theorem 15 that there is a bijection between each set of 
generalized oscillating tableaux GOT([n], -< q , A, I) with n > L q (£(X), Ai) and a set 
of vacancy-osculation sets. 

Theorem 15. Consider n G P, q G 1, a q-order -< q , A G Par, and I G N. Then for 
n < L q (£(X),Xi), GOT([ra], -< q , A, I) is empty, while for n > L q (£(X), Ai) , the map- 
ping $ g is a bijection from GOT([n], -< q , A, I) to VOS(a(n,q),b(n,q),a a ( ntq ) t b(n,q),\, 
Pa(n, q ),b(n,q),xJ) , where a(n,q) = max(n, n — q) and b(n,q) = max(n, n + q), and the 
inverse mapping is ®(L q ,^„)- 

Here, L q is defined in (11081) . and (a a (n, g ),6(n,g),A, Pa(n, q ),b(n,q),\) is the boundary point 
pair in BP(a(n, q), b(n, q)) which corresponds to A, as given explicitly by flol?]) . It 
can be checked straightforwardly that A G Par(a(n, q), b(n, q)), for n > L q (£(X), Ai), 
so that {a a {n, q ),b{n,q),X^a{n,q),b{n,q),x) is well-defined. Note also that (a(n,q),b(n,q)) 
is simply (n — q,n) for q < 0, or (n,n + q) for q > 0. It can be seen that if q 
and A are regarded as fixed and n as variable, then a(n, q), b(n, q), a a (n,g),6(n, g ),A and 
Pa{n,q),b(n,q),\ represent a sequence of rectangles and boundary points with a fixed 
difference q between the length and width of the rectangle, and a fixed corresponding 
partition A. This occurs, for example, in the first three cases of Table Q] if m is 
regarded as fixed and n as variable. 

Proof. For n > L q (£(X), Ai), the result follows immediately from Theorem 13. For 
n < L q (£(X), Ai), the case A ^ and < q < Ai— 1 will now be considered in detail, 
all other cases being similar. Assume that there exists (t, r/) G GOT([n], -< g , A, /). It 
can be seen, using (|89|) and the first two properties of fl90|) . that there exist integers 
1 < k < ki < . . . < k Xl - q -i < I and 1 < m < m\ < . . . < m^xyi < I such that 
^( r ?)fc 1 = Q+h for each i G [0, Ai — q— 1], and Q(r]) mi = —i, for each i G [0,£(X) — 1]. 
Since, by (11 101) . Ai — 1 -< q ... ~< q q+1 ~< q q, it follows that t^ > i + 1 for each 
i G [0, \\—q—l], and so that ifc Al _ 9 _ 1 > Aj— q. Also, — £(X)+1 -< q . . . -< q — 1 ^ g 0, from 
which it follows that t mi > i + 1 for each i G [0,£(X) — 1], and so that t me(xyi > £(X). 
But t kxi _^_ 1 < n and t^,^ < n must both hold, giving n > max(£(A), Ai— g), which 
contradicts n < L q (£(X),Xi) = max(£(A), Ai — q) (since q > 0), and implies that 
GOT([n], -< q , A, I) is empty. □ 
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Theorem 15 and (11041) give 



11 f n + i' 



i 



i=0 



I 



(118) 

0, n<L q {£{X),X x ) 

|OP(a(n,g),fe(n,g),a a ( n) g) )6 ( n> g) ) A,/3o(n > g) ) 6(n, 9 ),A,0l> n - L q (£(X),Xi). 

Since the RHS of (11181) is independent of the g-order -< q , it follows that the distri- 
bution of the number of ascents A on any fixed OP (A, I) is the same with respect to 
any g-order, i.e., if -< q and -<' both satisfy (1 11 01) for q G Z, then 

Af(XJ^ q ,t) = J\f(X,l, -<' q ,i) , (119) 

for any A G Par, / G N and ie [0,1-1]. 

It also follows from (11181) . the first equation of ( 1531) and the first bijection of (jHJ) that 

N{X,l,^ q ,i) = JV(A*,Z, -<- q ,i) , (120) 
for any A G Par, I G N, g G Z, g-order -< g , — g-order -<_ g and « G [0, 1 — 1]. 



15. Summary 

The demonstration that there exist bijections between tuples of osculating paths 
and generalized oscillating tableaux constitutes one of the main components of this 
paper. However, since the details of these bijections appear across many of the 
previous sections, they will be restated in this section as a summary. 

Starting with mappings from path tuples to generalized oscillating tableaux, consider 
a, b G P, / G N, and a boundary pair (a,/3) G BP(a, b), with BP (a, b) defined 
in ([3]). Then, for any total strict order -<b-a on Z which satisfies (IllOp for q = 
b — a, there is a bijection from the set OP(a, b, a, (3, t) of path tuples, as defined 
in (j7j) using fl2]) and fl§D, to the set GOT([min(a, b)], -<b-a, X a ^ a ^, I) of generalized 
oscillating tableau, as defined in fll03[) using f|45l) . (jSTl) . ([55]) and flH7|) . For any 
path tuple P G OP(a, 6, a, j3, 1), the corresponding generalized oscillating tableau 
(t, rj) G GOT([min(a, b)}, -<b-a, X a ,b,a,/3, 1) can be obtained as follows. 

• Find the set S of I points of [a] x [b] which are vacancies or osculations of P, 
i.e., points through which pass zero or two paths of P respectively. 
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• Order the points of S as the unique tuple s = ((«i, ji), . . . , which satis- 
fies ( 150]) . with L taken to be L&_a, as defined in fllQ8j) . and -< taken to be -<b-a- 

• Obtain the unique oscillating tableau T)(s) whose profile, as defined in (155)) . is 
given by f llOOjl . This step can be performed by simply starting with r/(s)o = 0, 
and then obtaining the Young diagram of each successive partition f](s)k from 
that of r](s)k-i by adding or deleting (only one or the other being possible) a 
square with content jk—h, for each fee [I]. 

• The generalized oscillating tableau (t,rj) is then (ii, ji), . . . , L^^i, ji)), 
r)(s)). 

In this mapping, for each k G [/], is a deletion of 77(5), i.e., a value for which 
|ry(s)fc| = \r)(s)k-x\ — 1, if and only if (ik,jk) is an osculation of P. 

For the example of Figure^ S", s, r/(s) and (t, 77) are given in (1751) . ([Mil and f)105|) 
respectively. 

Proceeding to mappings from generalized oscillating tableaux to path tuples, con- 
sider n G P, g G Z, a total strict order -< g on Z which satisfies ( II 101) . Z G N, and 
a partition A, with n > L g (£(X),Xi), where L q is defined in (11081) . Then there is 
a bijection from the set GOT([n], -< g , A, I) of generalized oscillating tableaux to the 
set OP(a(n,q),b(n,q),a a ^ q )^ ntq ) i x,f3a(n, q )Mn,q),xJ) of path tuples, where a(n,q) = 
max(n,n-g), b(n,q) = max(n,ra + g), and {a aM;bMjX , Pa{n,q),b(n,q),\) is defined 
using (1521) . For any generalized oscillating tableau (t, 7/) G GOT([n], A, I), the 
corresponding path tuple P G OP(a(n, g), b(n, q), a a (n, q ),b(n,q),^ Pa(n, q ),b(n,q),\i can 
be found as follows. 

• Obtain the profile £l(rj), as defined in (1551) using (1151) and (157)) . of the oscillating 
tableau 77. 

• Obtain the vacancy-osculation set S = 77), where $ g is defined in (1114j) 
using (fTT2D . 

• Use (136)) to obtain the edge matrix pair (H, V) corresponding to S. 

• The path tuple is then the unique P G OP(a(n, g), b(n, g)) which satisfies (1TB1) . 

For the example of flTUSD . Q(r]), S, {H, V) and P are given in Table El (1521). (TTTD and 
Figure [2] respectively. 

For two sets related bijectively by one of mappings described here, the other mapping 
is the inverse mapping, where the same total strict order on Z is used throughout. 
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16. Lexicographic and Antilexicographic Ordering Pairs 

In this section, the lexicographic and antilexicographic ordering pairs (L, <) and 
(L, >), with L and L defined in (1811) and (j82p . will be considered further, and it will 
be seen that in certain cases these lead to alternative bijections to those described 
in the previous section. 

By Theorem 11, for any a, b G P, (a, (3) G BP(a, b), and I G N, the mappings induced 
by (!/,<) or (L,>) are injective from VOS(a, b, a, f3, 1) to GOT([a],<,A a ^,i) or 
GOT([6], >, \ a ,b,a,/3, respectively. These mappings are not always bijective, but it 
will be seen in Theorem 16 that 0^ < - ) is bijective if a = [a] and that is 
bijective if (3 = [b]. Note that a = [a] implies that a < b, and that j3 = [b] implies 
that b < a. The partitions for these cases are given in (j5%|) . and it can be seen that 

• a = [a] if and only if (\ a) b, a ,p)i < b—a 

(121) 

• P = [b] if and only if d-{\ a ,b,afi) < a — b. 

Theorem 16. Consider a, b G P, (a, (3) G BP(a, 6), and I G N. If a = [a], then 
the mapping 0^ < n induced by the lexicographic ordering pair (L, <) is a bisection 
from VOS(a, b, [a], (3, 1) to GOT([a], <, \ a ,b,[a],/3, 0; an< ^ */ P = then the map- 
ping ©(£ >) induced by the antilexicographic ordering pair (L, >) zs a bijection from 
VOS(a, 6, a, [6], /) to GOT([6], >, X a> b,a,[b], 0- ^ e inverse mappings for these cases 
are respectively 

$f(t,rj) = {(t 1 ,t 1 +Q(ri) 1 ),...,(t h t l +n(ri) l )}, (122) 
/or eac/i (t,r?) G GOT([a], <, X a ,b,[a],/3, 1), and 

%(t,rj) = {(t 1 -n(r ] ) 1 ,t 1 ),...,(t l -n(r ] ) h t l )}, (123) 
for each (t t rj) G GOT([6], >, \ a ,b,a,[b]J) ■ 

Proof. The proof is similar to that of Theorem 13. Due to Theorem 11, it only 
needs to be shown that 0^ < ^ and 0(£,>) are surjective, with inverses \& and This 
will now be done explicitly for the case a = [a]. The case (3 = [b] follows from the 
case a = [a] by reflection of the lattice in the main diagonal using the first bijection 
of ©. 
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Consider any (t,rj) G GOT([a], <, \ a ,b,[a],i3, 1), and again abbreviate Q(i]) k as Q k for 
each ke[t\. By f fT03|) . 1 < t x < . . . < U < a, and for n, n' G [I], 

If n < n' and O n > f2 n /, then t n < t n > . (124) 

For each k G [/], define = (i k ,jk) by 

Sfc = (t k ,t k + Q k ) . (125) 

It will now be shown that £ [a] x [b] for each A; G [/]. It can be seen immediately 
that i k G [a], j k > 1 for > 0, and jfc < b for fi^ < 6 — a, so it only remains to 
be checked that j k > 1 for Q k < 0, and j k < & for Q k > b — a. If Q k < 0, then, by 
applying the first property of (MI]) — times in succession, it follows that there exist 
integers 1 < n < ri\ < . . . < n_Q k = k such that Q nm = —m for each m G [0, — il^]. 
It then follows using (I124p that t Hm > m + 1 for each m G [0, — Qk] and so, taking 
m = —Q k , that t k > — 0^ H- 1, i.e., by (11251) . that j k > 1 as required. Similarly, if 
Q k > b—a, then abbreviating (\ a ,b,[a],i3)i = {3 a — a as Ai, noting that Ai < b — a, and 
applying the fourth property of f[5Uj) Q k —X 1 times in succession, it follows that there 
exist integers k = ^Q fe _ Al < ■ ■ ■ < rii < n < I such that Q nm = m + Xi for each 
m G [0,^^ — Ai]. It then follows using (j!24p and t\ < a that t nm < a — m for each 
m G [0,Ofc— Ai] and so, taking m = Q k —\i, that t k < a—Q k +Xi, i.e., by fl!25j) . that 
jk < a + Ai < 6 as required. 

Having now verified that G [a] x [b], and observing from (l8Tj) and (I125P that 
L(sk) = t k and jfe— «fc = for each k G [/], it follows by Lemma 12 that {s\, . . . , s{\ G 
VOS(a, b, [a], (3,1) and that 6^ < - ) ({si, . . . , s^}) = (£,??), so that @(l <) is surjective, 
with the inverse mapping □ 

Corollary 17. Consider a,b G P ; («,/3) G BP(a,6) ; and / G N. If a = [a], then 



\OP(a,b,[a),P,l)\ = £ ( a + A ^ Tj) ], (126) 



and if (3 = [b] , then 



\OP{a,b,a,[b},l)\ = £ fb + A(>, V )\ (127) 

•j60T(A , 6ia , w ,I) V f / 

Proof. This follows immediately from Theorem 16, (I104p . and the bijection between 
OP(a, 6, a, (3, 1) and VOS(a, 6, a, /?,/). □ 
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It follows from (11261) that |OP(a, b, [a],/3, l)\ is independent of b. This can also be 
seen by examining the paths directly since, for any P G OP (a, b, [a], (3, 1) and k G [a], 
path Pj, must end with a segment from (k, (3 a ) to (fc, 6). Similarly, |OP(a, b, a, [b], l)\ 
is independent of a, as can be seen from (11271) or from the fact that, for any P G 
OP(a, b, a, [b], I) and k G [6], path P^ starts with a segment from (a, /c) to (a&, k). 

It also follows from ffTTTj) . ffml) . ffT26l) and ffT2TD that the distribution of „4 on 
OP (A, /) is the same with respect to < or any g-order -< q with q > Ai, and the same 
with respect to > or any g-order -< q with q < — £(X), 

Af(\,l,<,i) = Af(\,l,^ q ,i) for any q > Ai 

(128) 

Af(\,l,>,i) = Af(\,l,^ q ,i) for any q < -£(X), 
with any A G Par, / G N and i G [0, /-!]. 



17. Further Example 

In this section, a further example, that of enumerating OP(n,n, [n], [ji],6) for any 
n G P, will be considered. 

As seen in Section 4, the path tuples of OP(n, n, [n], [n]) correspond to nxn standard 
alternating sign matrices, and, as seen in Table [TJ, X n ,n,[n},[n] = 0- By flo^j) . any 
path tuple of OP(n,n, [n], [n]) has equal numbers of vacancies and osculations, so 
OP(n, n, [n], [n], 6) is the set of all path tuples of OP(n, n, [n], [n]) which have 3 
osculations and 3 vacancies. 

In this case, (11171) . (11261) or (11271) can be used, giving 

'n + A(^o, r))\ 



|OP(n, n, [n],[n], 6)| = £ 



r/GOT(0,6) V " 



»7eOT(0,6) V ^ / ^gOT(0,6) V 6 y 



(129) 



where -< is an y 0-order. 

By f l99|) . |OT(0,6)| = 5!! = 15. These 15 oscillating tableaux are shown in Table [51 
In this table, the Young diagrams of each partition of rj, the terms of the profile 
of rj, with f2(r])jt abbreviated as Qk for each k G [6], and the sets of ascents A(-< , rj), 
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A(<,rj) and A(>,rj) are shown for each r\ G OT(0,6), where -<o is taken to be the 
specific 0-order of fillip . 

z -< z ' if an d only if \z\ > \z'\ or z = —z' < , (130) 

for z, z' G Z. 
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001 10 




i j j 


151 


□ □ □ □ 

l_l 


-1 -1 


{5} 


{5} 


{3} 


(h n m n m n (h 

V 1 1 1 1 1 1 1 1 1 1 1 1 \J 


11110 


151 


11 \ 


151 


□ g □ g □ 


-1 -1 -1 -1 


{5} 


{5} 


{1} 


□ m □ g □ 


1 1-1-10 


{5} 


{1,5} 


{3} 


□ g □ m □ 


0-1-1 1 10 


{3,5} 


{3} 


{1,5} 


□ m m m □ 


1-1-1 10 


{4,5} 


{1,4} 


{2,5} 


□ g gp g □ 


0-1 1 1-10 


{2,5} 


{2,5} 


{1,4} 


□ m gp g □ 


1-1 1-10 


{3,5} 


{1,3,5} 


{2,4} 


□ g gp m □ 


0-1 1-1 10 


{2,4,5} 


{2,4} 


{1,3,5} 


U LU 1 1 1 1 LU u 


12 2 10 


{4,5} 


{1,2} 


{4,5} 


□ g g g □ 


-1 -2 -2 -1 


{4,5} 


{4,5} 


{1,2} 



Table 5: The oscillating tableaux of shape and length 6. 



54 



It can be seen from Table [5] that 



JV(0, 6, -< Q , i) = AA(0, 6, <, z) = jV(0, 6, >, z) 



n, i = o 

7, i = l 

6, i = 2 

1, z = 3 

k 0, otherwise. 



so using this in H129[) gives 
|OP(n,n>]>],6)| = 



n 
6 



+ 7 



n+1 
6 



+ 6 



n+2 
6 



6 



(131) 



(132) 



Finally, note that in addition to the equalities between the three distribution func- 
tions of ffTBTD . ffT28|) also gives A/"(0, 6, -<„ i) = JV(0, 6, <, i) = JV(0, 6, >, i) for any 
g-order -< q , g6Z and z G [0, 5]. 



18. Nonintersecting Paths and Semistandard Young Tableaux 

In this section, the special case of tuples of nonintersecting paths in an a by b rect- 
angle, with the paths starting and ending at points (specified by (a,/3) G BP(a, b)) 
along the lower and right boundaries, and taking only unit steps rightwards or up- 
wards, will be considered. It will be seen that, using the lexicographic ordering 
pair, there is a bijection between such path tuples and certain semistandard Young 
tableaux. Three different enumeration formulae will then be obtained, the deriva- 
tions of two of which involve using the Lindstrom-Gessel-Viennot theorem. 

A tuple of nonintersecting paths, P G OP(a, b, a, (3), is a path tuple which has 
no osculations, i.e., X(P) = 0, X {P) = and Z(P) = N{P), and so, by (|59lh 
|Z(P)| = \\ a ,b,a,/3\- Accordingly, the set of tuples of nonintersecting paths in [a] x [b] 
with boundary point pair (a, 0) G BP (a, 6) is defined as 

NP(a,6,a,/3) := OP(a, b, a, 0, \X a ,b,aA) ■ ( 133 ) 

It also follows from (llOlj) that for any S G VOS(a, b, a, j3, \X a ,b,a,i3\) and any progres- 
sion s for S, the oscillating tableau rj(s) associated with s has no deletions. There- 
fore, as indicated in Section 11, rj(s) corresponds to a standard Young tableau. Of 
primary interest here will be the progression s = . . . , for S induced 

by the lexicographic ordering pair (L,<) (i.e., the lexicographic ordering of S), 
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where I = \\ a ,b,a,p\- m this case, the generalized oscillating tableaux associated 
with S via (L, <) is, using §2) and ffT06|) . Q { i,<)( S ) = ((^( s i)> • • • > ^O)) 
= ((ij, . . . , z,), r?(s)) G GOT([a], <, A tt)6 , a ^, Z). 

An example of a tuple P of nonintersecting paths in NP(4, 6, {1, 2, 4}, {1, 4, 5}) is 
shown diagrammatically in Figure [7J The progression for the vacancy-osculation 



Figure 7: Example of a tuple of nonintersecting paths. 

set S of P induced by the lexicographic ordering pair (i.e., the lexicographic ordering 
of the vacancies of P) is s = ((hji), (isjs)) = ((1, 1), (1,2), (1,3), (2, 1), (3, 1), 
(3, 3), (3, 6), (4, 3)). It can be seen that (ji—i%, ■ ■ ■ , js - is) — (0, 1, 2, —1, —2, 0, 3, — 1) 
matches the profile of the oscillating tableau of Table HI implying that this is the 
oscillating tableau r)(s) associated with s, and that the corresponding standard 
Young tableau is a of fl98|) . The generalized oscillating tableau associated with 
S via (L, <), (SO = ((ii, • • -,i&), r)(s)), is therefore 



(t,v) 



1,1,1,2,3,3,3,4), 

(0, (1), (2), (3), (3, 1), (3, 1, 1), (3, 2, 1), (4, 2, 1), (4, 2, 2))' 



(134) 



For a subset T of P, and a partition A, define the corresponding set of semistandard 
Young tableaux as 

SSYT(T,A) := { r • r is an array of entries G T with (i,j) G Y(A) 

• Tij < r i,ff-i fo r each i G [^(A)], j G [Aj — 1] (135) 

• Tij < r mj for each % G [£(\)-l], j G [A w ]} . 

As with standard Young tableaux, r G SSYT(T, A) is usually shown by writing 
each nj within the unit square centered at (i,j) in the Young diagram of A. These 
entries now increase weakly along each row and increase strictly down each column. 
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It can be checked that, for any jigP and partition A, each generalized oscillating 
tableau (t, rf) G GOT([n], <, A, |A|) can be bijectively associated with a semistandard 
Young tableau r G SSYT([n],A) where, for each k G [|A|], if Y(rj k ) = Y(r] k _i) U 
then = t k . For the inverse mapping from r to [t,7]), the points of 
Y(A) should first be ordered as ■ ■ ■ such that r ik j k < r ik+1 j k+1 , or 

r *fcifc = r Wfc+i and Jk-ih < j'fc+i — ifc+i , for each k G [|A|-1]. Then (t,rj) is given 
by t k = r ik j k and fl(r]) k = j k — ik for each k G [|A|]. For example, the semistandard 
Young tableau which corresponds to the generalized oscillating tableau of (I134p is 



1 


1 


1 


3 


2 


3 




3 


4 



(136) 



It follows from this bijection between GOT([n], <, A, |A|) and SSYT([n], A), the bi- 
jection between OT(A, |A|) and SYT(A), and fll04p that 



SSYT(M,A)| = 




(137) 



where A is the number of ascents for standard Young tableaux, as defined in Sec- 
tion 11. This result, which could have been derived directly rather than by using 
oscillating tableaux, is contained in [541 Proposition 7.19.12]. Note that there also 
exists a simpler formula for |SSYT([n], A)|, namely the hook- content formula (see 
for example [541 Corollary 7.21.4]). 

Now define, for any S G VOS(a, b, a, j3, \X a ,b,a,/3\)i T(5) to be the semistandard 
Young tableau associated with the generalized oscillating tableau 6^ <)(£). Thus, 
for the vacancy-osculation set S corresponding to the path tuple of Figure [TJ Y(S') 
is given by ( I136p . 

Also define, for any a, b G P and A G Par, 

SSYT(a, b, A) := {r G SSYTQa], A) | r lX < b-Xi+i for each % G [£(X)]} . (138) 

Note that the condition t^x. < b—Xi+i places an additional constraint on the largest 
(i.e., rightmost) entry of row i of r whenever A.; — i > b—a. 

By Theorem 11, T is an injective mapping from VOS(a, b, a, j3, |A ai & ia)i g|) to SSYTQa], 
X a ,b,a,p), and it can be shown using Lemma 12 and arguments similar to those 
used in previous sections, that T is a bijection from VOS(a, 6, a, (3, \X a ,b,a,i3\) to 
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SSYT(a, b, X a ,b,a,p)- Also, by Theorem 16, if a = [a], then SSYT(a, b, X a ,b,[a],/3) = 
SSYT([a],A aAH/3 ). 



Note too that in addition to T, which involves the lexicographic ordering pair 
and gives a bijection from VOS(a, b, a, (3, \\ a ,b,a,p\) to a set of semistandard Young 
tableaux, there is also an analogous bijection which involves the antilexicographic 
ordering pair, and gives a bijection from VOS(a, b, a, /3, lA^o^l) to a set of row- 
strict Young tableaux (i.e., arrays whose entries increase strictly along each row and 
increase weakly down each column). 

It now follows using the bijection T and Corollary 14 that 



NP{a,b,a,/3)\ = |SSYT(a, b, X aM )\ = 

f min(a, b) + A(^, v) \ (139) 



for any (b — a)-order. The sum over oscillating tableaux in (11391) can instead be 
regarded as a sum over standard Young tableaux SYT(A ai ft jQ , 5j g). For the case a = 
[a], it follows that |NP(o, b, [a], (3)\ = |SSYT([a], X a) b ; [a] } p)\, so NP(a, b, [a], /3) can 
alternatively be enumerated using (I137P or the hook-content formula. 

The Lindstrom-Gessel-Viennot theorem (see for example [Ml Theorem 1], [251 Corol- 
lary 2] or [53], Theorem 2.7.1]) states that the number of tuples of nonintersecting 
paths with certain fixed endpoints can be expressed as the determinant of a matrix 
of binomial coefficients. For r paths, i.e., \a\ = \/3\ = r, this theorem then gives an 
alternative formula for |NP(a, b, a, f3)\ as the determinant of an rxr matrix, 



where as usual the elements of a and (3 are labeled in increasing order. 

Finally, as indicated in Section 1, semistandard Young tableaux can also be as- 
sociated with tuples of nonintersecting paths using a method in which each row 
of a tableau is regarded as a partition, by reading the row's entries from right to 
left, and then associated with the path formed by the lower and right boundary 
edges of the Young diagram of that partition. By translating these paths appro- 
priately on the lattice, this gives a bijection between SSYT(a, b, X a ^ t0l ^) and a cer- 
tain set of £(A ai f, iQj /3)-tuples of nonintersecting paths, which is therefore in bijection 
with NP(a, b, a, (3). Using [25l Theorem 3], which combines this bijection with [25l 






i=a— ai,...,a-o ri j=b—/3i,...,b—(3 r 



(140) 
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Corollary 2], then gives a third formula for |NP(a, b, a, (3) \ as the determinant of an 

Z{K,b,a,i3) x^(A a , 6) a )( g) matrix, 

|NP(a,6,a,/3)| = 

(A a ,fe,a,/3)i-^+J+ min (^-(AaAa,/3)i + ^ a )- 1 \ \ (141) 

19. Discussion 

All of the results of this paper have now been presented, so in this final section, the 
scope for additional, related work will be discussed. 

Some possible directions for further research are: 

(1) Studying the combinatorics of osculating paths with external configurations 
other than fixed start and end points on the lower and right boundaries of a 
rectangle. 

(2) Investigating whether the generalized oscillating tableaux defined in this paper 
can be interpreted within a representation theoretic context. 

(3) Applying further bijections to generalized oscillating tableaux, with the aim of 
obtaining bijections between osculating paths and other known combinatorial 
objects. 

With regard to (1), some obvious possibilities for other external configurations are 
fixed start points on the lower and left boundaries, and end points on the up- 
per and right boundaries, of a rectangle, boundary conditions which correspond to 
previously-studied subclasses of alternating sign matrices other than those outlined 
in Section 4 (see for example [36l HI]), boundary conditions which correspond to 
previously-studied cases of the six-vertex model of statistical mechanics (see for ex- 
ample [H El EH 1121 ED Ell), an d the configuration used in studies of friendly walkers 
in which the paths start and end on parallel lines at 45° to the rows or columns of 
the lattice (see for example [3 El M, M, El] ) • 

With regard to (2), oscillating tableaux have often been studied in the context 
of the representation theory of the symplectic group or Brauer algebra (see for 
example [H [161 1131 E3 EH EQ]). Certain generalizations of oscillating tableaux 
have also been introduced (see for example (131 [HI [T71 [231 E2l EH]), but these all 
appear to differ from the generalization used in this paper. It would therefore 
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seem worthwhile to investigate whether the generalized oscillating tableaux used 
here can be related to representation theory. Further motivation for searching for 
representation-theoretic interpretations of aspects of this paper is that it is shown 
in [H] that enumeration formulae for standard alternating sign matrices, and various 
subclasses thereof, can be obtained using determinantal formulae which are related 
to characters of irreducible representations of classical groups. 

With regard to (3), it would seem natural to combine the bijections obtained in this 
paper, between tuples of osculating paths and generalized oscillating tableaux, with 
further bijections involving oscillating tableaux. Certain such bijections, which can 
be regarded as analogs of the Robinson-Schensted-Knuth correspondence (see for 
example [22], [52] and [Ml Ch. 7]), are already known (see for example [6l [EH [HI 
Uni HZl S3l SH S3 EDI EZl [SHI), although these need to be generalized in order to 
be applicable to the generalized oscillating tableaux being used here. It is expected 
that the results of such work will be reported in a sequel to this paper. In particular, 
it is hoped that it will be possible to exhibit bijections between certain tuples of 
osculating paths and certain tuples of nonintersecting paths. 

In conclusion, an example of such a result, and its consequences, will now be out- 
lined. For any n G P and I G N, there appears to be a bijection between the set of 
tuples of osculating paths OP(n, n, [n], [n], 21), and the set of tuples of nonintersect- 
ing paths UA N z,Ai<nNP(n,n,Q! ni „ )D ( A ) ) /3 rii „ )D ( A )), where (J52J) is being used, U AMjAl<n 
denotes the union over all strict (i.e., with distinct parts) partitions A of / with largest 
part less than n, and -D(A) is the double of A, defined using Frobenius notation to 
be the partition (Ai, . . . , A r | Aj— 1, . . . , A r — 1), where r is the length of A. Note that 
OP(n, n, [n], [n], 21) corresponds to the set of nxn standard alternating sign matrices 
whose corresponding path tuple has I vacancies and I osculations. Such a bijection 
seems related to the analog of the Robinson-Schensted-Knuth correspondence de- 
scribed in [12l Sec. 4]. Applying (11401) to the individual sets of nonintersecting paths 
gives a formula for |OP(n,n, [n], [n],2l)\ as a sum of determinants, and then sum- 
ming over all possible values of I, with weight x l for a parameter x, and combining 
these determinants into a single determinant, gives 



Setting x = 1 gives a determinantal formula for \OP(n,n, [n], [n])\, or equivalently 



n(n-l)/2 



]T \OP(n,n,[n],[n],2l)\x' 



1=0 



(142) 
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the number of n x n standard alternating sign matrices, which was obtained in [26l 
Remark 5.2] by transforming the determinant used in [T] for the enumeration of 
descending plane partitions. However, it is hoped that the method being used here 
will clarify the underlying combinatorics. 
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